TokenPad

Recipe · Cost

Estimate cost before sending

Price a request client-side before dispatch, so you can budget, warn or block — with the caching and reasoning adjustments most estimates omit.

The problem

You want to know what a request will cost before making it — to enforce a per-user budget, to warn on an expensive operation, or simply to have the number in your logs.

A naive estimate misses two things that change the answer substantially: cached input bills at a fraction of the rate, and reasoning models generate output you never see.

Price a request with caching accounted fortypescript
interface Pricing {
  /** USD per 1M tokens. */
  input: number;
  cachedInput?: number;
  output: number;
}

interface Estimate {
  inputTokens: number;
  cachedTokens: number;
  expectedOutputTokens: number;
}

export function estimateCost(pricing: Pricing, e: Estimate): number {
  const uncached = Math.max(0, e.inputTokens - e.cachedTokens);
  const cachedRate = pricing.cachedInput ?? pricing.input;

  return (
    (uncached * pricing.input +
      e.cachedTokens * cachedRate +
      e.expectedOutputTokens * pricing.output) /
    1_000_000
  );
}
Enforce a budget before dispatchtypescript
const MAX_COST_PER_REQUEST = 0.05;

export async function guardedComplete(messages: Message[], pricing: Pricing) {
  const inputTokens = countMessages(messages);

  // The stable prefix — system prompt and tool schemas — is what caching serves.
  const cachedTokens = countMessages(messages.filter((m) => m.role === 'system'));

  const estimate = estimateCost(pricing, {
    inputTokens,
    cachedTokens,
    expectedOutputTokens: 500, // your measured p50, not max_tokens
  });

  if (estimate > MAX_COST_PER_REQUEST) {
    throw new BudgetExceededError(
      `Estimated ${estimate.toFixed(4)} exceeds the ${MAX_COST_PER_REQUEST} ceiling.`,
    );
  }

  return client.chat.completions.create({ model, messages, max_tokens: 1024 });
}
Reconcile against what was actually billedtypescript
const response = await client.chat.completions.create({ /* ... */ });

// The usage field is the truth. Log both so you can calibrate the estimate —
// a consistent gap usually means an assumption is wrong, not the API.
const actual = {
  input: response.usage?.prompt_tokens ?? 0,
  cached: response.usage?.prompt_tokens_details?.cached_tokens ?? 0,
  output: response.usage?.completion_tokens ?? 0,
  reasoning: response.usage?.completion_tokens_details?.reasoning_tokens ?? 0,
};

logger.info('llm.cost', { estimated: estimate, actual, model });

Why it is written this way

Use your measured p50 for output, not max_tokens

max_tokens is a ceiling you set, not a prediction. Estimating with it overstates cost by a large factor and makes the guard useless — everything looks expensive.

Reasoning tokens are billed and invisible

On reasoning models the internal thinking is charged at the output rate and does not appear in the response. Where the usage field reports it, add it to your estimate rather than a multiplier.

Reconcile, do not assume

Log the estimate beside the actual usage. A consistent gap tells you which assumption is wrong, and it is usually the cached share or the output length.

What breaks the naive version

  • Ignoring cached input. On a workload with a stable prefix it is most of the input line, and pricing it at the base rate makes every estimate wrong.
  • Counting only the user message rather than the assembled request.
  • Estimating in the same code path that sends the request without caching the token count — you encode the same text twice.

Check your numbers