Your AI feature works well with ten test requests. Then one customer uploads a large document, presses retry, or starts an automated loop. The feature still works, but the bill no longer matches your plan.
Control cost inside the product, not only in the provider dashboard. Measure one customer result, limit the work each request can start, and decide what the app should do when a boundary is reached.
What are you paying for?
AI providers measure work in small units. Text models often count pieces of input and output called tokens. Images, audio, web search, stored files, and other tools may have separate charges.
The same feature can cost more when it sends a longer history or accepts a larger file. Longer answers, retries, and more capable models can add more work. Exact prices change. Check the current OpenAI pricing, Anthropic pricing, or Gemini API pricing for the provider you use.
Which number should you measure?
Do not stop at “cost per API call.” A customer may need several calls to finish one useful task. Measure the cost of a successful customer result.
For each AI feature, record:
- The feature and customer account
- The model and provider
- Input and output units reported by the provider
- Extra tool, image, audio, or search use
- Whether the customer result succeeded
- The estimated provider cost
Do not copy private prompts or customer documents into cost logs. Keep only the numbers and safe identifiers needed to explain the bill.
How do you find the expensive path?
Group cost by feature, customer, model, and result. OpenAI's Usage and Costs APIs, for example, can group activity by project, model, user, or API key and report billed cost.
Look for four patterns:
- A few customers use much more than the rest.
- A failed task retries several times.
- Every request sends old text the model no longer needs.
- A simple task always uses the most expensive model.
Fix the largest repeated path first. A small saving on the task customers run every day matters more than a perfect one-time prompt.
Which boundaries should the app enforce?
| Boundary | What it prevents | What the customer sees |
|---|---|---|
| Input size | Huge documents or histories | A clear size limit before upload |
| Output size | Answers that run far past the useful result | A shorter result or follow-up option |
| Requests per account | Loops and accidental repeated clicks | Current use and when it resets |
| Retry count | One failure multiplying into many calls | A safe retry or support path |
| Concurrent work | Many expensive jobs starting together | Queued work and progress |
A dashboard alert tells you that money was spent. An app boundary can stop more work before it starts.
Should every task use the best model?
No single model is best for every step. Use a smaller or less expensive model for a narrow task when it meets your quality rule. Keep a stronger model for work that clearly needs it.
Test the change on real examples. Compare the customer result, failure rate, response time, and cost. Do not switch models only because the listed price is lower.
You can also shorten repeated instructions, send only relevant records, and reuse safe results that have not changed. Never reuse one customer's private result for another customer.
What should happen when a limit is reached?
The app should fail in a controlled way. It must not spin forever, hide the error, or keep charging while showing a blank screen.
- Stop the current work at the defined boundary.
- Keep the customer's original data safe.
- Explain which limit was reached in plain language.
- Say when they can retry or how to reduce the request.
- Give support a tracking number without exposing private content.
Provider account limits are useful, but they are not your whole product rule. Billing data can arrive late, and one account may contain several apps. Google notes this delay for its Gemini API spend caps. Keep a product-level boundary too.
How do you test the worst case?
- Use the largest allowed input.
- Request the longest allowed output.
- Press the action twice.
- Make the provider time out.
- Trigger every allowed retry.
- Run the maximum number of jobs at once.
- Confirm the final cost estimate and customer message.
Run this in a test project with a low provider limit. Never use private customer files for a cost test.
What should you ask your coding AI?
Copy this request: Inspect every place this app calls an AI provider. Do not change code yet. For each feature, list the model, input source, output limit, extra tools, retry behavior, timeout, concurrent jobs, and usage data we save. Estimate the normal and worst allowed work per successful customer result. Propose app-level limits, customer messages, and pass-or-fail tests. Do not include private prompt content in logs.
The answer should name every call site and every retry. If the AI cannot find the billing unit for a tool, mark it unknown and link the current provider documentation.
When is cost control ready?
- You can see cost by feature and customer result.
- Input, output, retries, and concurrent work have firm limits.
- A failed request cannot retry forever.
- The customer sees a useful message when a limit is reached.
- Your first price still covers normal and heavy allowed use.
- Someone reviews usage and alerts on a regular schedule.
What should you read next?
- Where Should API Keys Go? A Plain-English Guide
- Pricing an AI App Before You Have Many Customers
- Make Your Coding AI Test Its Work
When the limits are clear, visit JustDeploy to publish the app and inspect its live behavior.