Cost
Cached input
Cached input is prompt content the provider has already processed and retained, billed at roughly a tenth of the base input rate.
Also written: prompt caching, cache read
Caching is a prefix cache: the match must start at the very first character of the request and run contiguously. Writing to the cache costs a small premium — commonly 1.25× base for a short-lived entry — and reading from it costs about 0.1×, so a single cache hit already puts you ahead.
On a workload with a stable system prompt and tool definitions, most of the input line of the bill can move to the cached rate.
In practice
Cached reads run at roughly a tenth of base input rate on the providers that offer them. On a 4,000-token system prompt at a million requests a month, that is the difference between about $12,000 and about $1,200 — from a configuration flag. The catch is that one dynamic value near the top, a timestamp or a request id, invalidates the entire prefix after it.
Common questions
How much does prompt caching save?
Cached reads run at roughly a tenth of base input rate on providers that offer them. On a 4,000-token system prompt at high volume that is usually the single largest saving available, and it is a configuration flag rather than a rewrite.
Why is my cache hit rate zero?
Almost always something dynamic near the top of the prompt — a timestamp, a request id, a date, a user name. Caching matches on prefixes, so one changing value invalidates everything after it.
Measure it
- Prompt Cache Structure CheckerFinds cache-busting content and measures the prefix that survives it.
- LLM API Cost CalculatorRequests per month in, dollars out. Input, cached input and output priced separately.
- Chatbot Conversation Cost CalculatorHistory is resent every turn, so cost grows quadratically. Most budgets miss this entirely.