Traditional feature estimation asks how long it takes to build. AI features need a second estimate that conventional planning has no habit for: what it costs to run, every day, forever, scaling with adoption. Teams that skip it ship a feature that works and then discover the unit economics do not.
The four cost components
- Build — engineering time, the familiar part.
- Inference — tokens per request multiplied by requests, which is the part that grows with success.
- Retrieval infrastructure — embedding generation, vector storage, re-embedding when models change.
- Evaluation and monitoring — building the eval suite, and running it continuously against production traffic.
Estimating inference before you have built it
You can get within a useful range without writing the feature. Take a realistic example request, write the prompt you expect to send including retrieved context, and count the tokens. Then estimate output length. That gives you cost per request; the rest is arithmetic against expected volume.
Cost per request = (input_tokens x input_rate) + (output_tokens x output_rate)
Requests per month = active_users x requests_per_user_per_month
Monthly inference = cost_per_request x requests_per_month x retry_factor
Worked example, 5,000 active users, 20 requests each:
input 3,000 tokens (system + retrieved context + question)
output 400 tokens
retry_factor 1.15 (failures, regenerations, agent loops)The multiplier people forget is loop count
For agentic features, one user request is not one model call. A task that takes four tool-calling loops costs roughly four times a single-shot request. Loop count is emergent — it depends on prompt and tool quality — so estimate a range and instrument it from day one rather than assuming one.
Estimate the range, not the number
A single figure invites false confidence. Produce three: a base case, a case where usage is 3x higher than expected, and a case where average loop count doubles because the task is harder than modelled. If the pessimistic case is unaffordable, that is a design constraint discovered now rather than after launch.
The design decisions that dominate the number
- How much context you retrieve. Ten chunks instead of three roughly triples input tokens on every call.
- Which model tier handles which step. The largest single lever, often 10-30x on the steps that can move down.
- Caching. Repeated system prompts and stable context can often be cached, which changes the input economics materially.
- Whether output is streamed and stopped early. Users abandoning a long generation still pay for the tokens produced.
What to track once it ships
Attribute cost per feature and per task type, not just a monthly platform total. An aggregate bill tells you spend went up; per-task attribution tells you which change caused it. Tie this into the trace so a cost regression points at a span.
The estimate is a document, not a number
Record the assumptions — token counts, expected volume, loop count, model tier per step — alongside the figure. When actual cost diverges, the assumptions tell you which one was wrong, which is the only way the next estimate improves.
Frequently Asked Questions
How accurate can a pre-build estimate be?
Within a factor of two if you count tokens against a realistic prompt, which is enough to answer the only question that matters early: is this affordable at expected scale.
Should I include the cost of evals in the estimate?
Yes. Continuous evaluation against production traffic is an ongoing inference cost, and it is the one most often left out of the model entirely.
What about the cost of a model price change?
Treat provider pricing as a variable, not a constant. Anything whose viability depends on current rates staying fixed should be flagged as a risk in the estimate.
References
- How to Choose Between Small and Frontier Models — Towards Data Science
- 7 Agentic AI Trends to Watch in 2026 — Machine Learning Mastery
About Jishu Labs
Jishu Labs is a software development company founded in 2016. We build custom software, AI/ML systems, and full-stack web and mobile applications for clients, and we make eight AI tools for software teams.