TokenPad

Recipe · Reliability

Handle stop reasons properly

A truncated answer arrives as a successful request. How to detect it, what each stop reason means, and why checking is twenty lines that prevent a silent product defect.

The problem

When generation hits max_tokens the API returns success with an incomplete answer. There is no error, nothing in your logs, and the cut-off text renders to a user as if it were finished.

This is the most common silent failure in LLM applications, and it is trivial to detect.

Branch on the stop reason, not on exceptionstypescript
type Outcome =
  | { kind: 'ok'; text: string }
  | { kind: 'truncated'; partial: string }
  | { kind: 'refused'; text: string }
  | { kind: 'tool_call'; calls: unknown[] };

export function classify(response: {
  choices: { message: { content: string | null; tool_calls?: unknown[] }; finish_reason: string }[];
}): Outcome {
  const choice = response.choices[0];
  const text = choice.message.content ?? '';

  switch (choice.finish_reason) {
    case 'stop':
      return { kind: 'ok', text };
    case 'length':
      // Success at the HTTP level, incomplete in every way that matters.
      return { kind: 'truncated', partial: text };
    case 'content_filter':
      return { kind: 'refused', text };
    case 'tool_calls':
      return { kind: 'tool_call', calls: choice.message.tool_calls ?? [] };
    default:
      return { kind: 'ok', text };
  }
}
Validate structured output before trusting ittypescript
const outcome = classify(response);

if (outcome.kind === 'truncated') {
  // Truncated JSON is not shorter JSON — it is invalid JSON.
  // Repairing it in the parser hides a max_tokens problem.
  throw new Error('Response truncated: raise max_tokens or shorten the request.');
}

if (outcome.kind !== 'ok') {
  return handleNonAnswer(outcome);
}

const parsed = JSON.parse(outcome.text);
// Parsing is not enough — a valid object missing a field breaks further
// downstream, where the cause is much harder to find.
if (typeof parsed.category !== 'string') {
  throw new Error('Response parsed but is missing "category".');
}

Why it is written this way

Providers name it differently

OpenAI returns finish_reason with values like stop and length. Anthropic returns stop_reason with end_turn and max_tokens. The meaning maps cleanly, but code ported between them without changing the field silently stops detecting truncation.

A refusal is also not an error

A content filter or a model declining arrives as ordinary text where the answer should be. If your code only branches on exceptions it treats a refusal as a valid answer and shows it to a user.

Reasoning models make this worse

Internal thinking counts towards max_tokens. A ceiling sized for the visible answer can be exhausted before the answer starts, producing an empty response with a length stop reason.

What breaks the naive version

  • Assuming a 200 response means a complete answer.
  • Growing a JSON repair function to handle truncated objects, instead of raising the ceiling.
  • Logging only errors. A truncation rate is a metric worth watching, and it is invisible if you only count exceptions.

Check your numbers