LLM Cost Control Checklist

Each of these is a configuration change, not a redesign.

• Prompt caching is enabled and verified to be hitting.critical

  A cached prefix read costs a fraction of base input. For any application with a substantial system prompt this is the largest single-line saving available, and it is a flag.

  Check this with the prompt cache checker →
• Nothing dynamic appears before the cacheable portion of the prompt.critical

  One timestamp at the top means a nought percent hit rate while every dashboard reports caching as enabled.

  Check this with the prompt cache checker →
• max_tokens is set to the longest legitimate answer, on every call.

  Output costs several times what input does. Unbounded output is unbounded spend on the expensive side.

  Check this with the max tokens planner →
• The prompt asks for the length you actually want.

  Models default to thorough. "Answer in at most three sentences" is a cost control instruction as much as a style one.

• The system prompt has been read line by line in the last quarter, and the lines that stopped mattering are gone.

  Prompts accrete permanently and are trimmed rarely. Every request pays for every line, forever.

  Check this with the prompt optimizer →

• Easy requests go to a cheap model and only hard ones reach the expensive one.critical

  Classification, extraction and routing rarely need a frontier model. The price difference between tiers is often more than tenfold, and most traffic is easy.

  Check this with the model routing savings →
• Anything that does not need an answer within the hour goes through the batch API.

  Batch pricing is typically half. Evaluation runs, backfills, summarisation of yesterday's data and classification of a backlog all qualify.

  Check this with the batch savings calculator →
• Conversation history is trimmed or summarised rather than sent whole.critical

  Sending the entire transcript on every turn makes cost grow quadratically with conversation length. The twentieth message costs twenty times the first.

  Check this with the chat cost estimator →
• Large documents are retrieved from, not pasted in.

  A long context window is a capability, not an instruction. Sending 200,000 tokens to answer a question about one paragraph pays for 199,000 tokens of nothing.

  Check this with the context rot visualizer →
• Identical requests are answered from your own cache instead of the provider.

  Provider caching discounts the prompt. Your cache costs nothing at all when the question and the context are the same as five minutes ago.

• You know whether reasoning tokens are being billed, and how many.

  Reasoning models charge for tokens you never see. On some workloads the invisible tokens outnumber the visible ones.

  Check this with the reasoning token estimator →

• Spend is attributed to features, not just to an account total.

  A total tells you the bill grew. Attribution tells you which of last month's six changes did it.

• You know the cost per user, per conversation or per document — a unit that scales with the business.critical

  Total spend is expected to grow. Unit cost growing is the actual problem, and it is invisible in the total.

  Check this with the cost per user calculator →
• The cost distribution is known at p99, not only the average.

  The mean hides the long documents. A small share of requests routinely accounts for a large share of spend.

• A daily spend threshold raises an alert.critical

  Monthly billing means a runaway loop has thirty days to run. A daily check bounds the damage at one.

• Prices are re-checked quarterly against current provider pages.

  Model prices fall, and new tiers appear. A routing decision made a year ago was made against prices that no longer exist.

  Check this with the model finder →

• User input is truncated to a hard token ceiling before it is sent.critical

  The most expensive request you will ever serve is the one someone pasted a book into.

  Check this with the token counter →
• A per-user daily spend or request cap exists.

  Without one, a single automated client consumes the budget of every other user.

• Anything that calls a model in a loop has a maximum iteration count.critical

  The classic overnight bill is a retry loop with no cap meeting an error that never resolves.

  Check this with the agent loop simulator →
• Development and staging have their own keys with their own limits.

  A test that runs in a loop should not be able to spend the production budget.