Context length exceeded

What it means

The context window is one budget covering everything in the request — the system prompt, every message in the conversation, tool definitions, attached documents — plus the tokens the model is allowed to generate in reply. You have exceeded that total.

The number in the error is almost always larger than people expect, because they were counting only the part they wrote. In an agent loop or a retrieval system, what you wrote is frequently a small minority of the request.

Why it happens

Most likely cause first.

1. 1

   Conversation history grew unbounded

   The API is stateless, so every turn resends the entire history. A conversation that runs long enough will hit the window eventually, and it does so abruptly — the turn before was fine.

2. 2

   Retrieved context is larger than you think

   Eight chunks of what you assumed were 300 tokens can be 600 each if the chunker split on characters rather than tokens. Retrieved passages usually dominate a RAG prompt.

3. 3

   max_tokens plus input exceeds the window

   The input alone fits, but the ceiling you requested for the reply does not fit alongside it. OpenAI’s message breaks this down explicitly — note the "in the completion" figure.

4. 4

   Tool definitions are counted and were forgotten

   Every tool name, description and JSON schema is part of the request. A dozen tools routinely exceeds a thousand tokens before the user says anything.

How to fix it

1. 1

   Measure what you are actually sending

   Not the user message — the assembled request. Paste it whole and get the real figure before changing anything, because the cause is usually the part you were not counting.

   LLM Token CounterReal BPE tokenization, not characters ÷ 4. Shows which counts are exact and which are estimates.
2. 2

   Check the model can hold it at all

   Some models have a 200K window and others a million. If your input is genuinely that large, the fix may be a different model rather than a smaller prompt.

   Context Window CalculatorPaste your context. See which models swallow it whole and which will reject the request.
3. 3

   Reserve output space explicitly

   Decide the longest reply you need, subtract it, and treat the remainder as your real input capacity. Filling a window to the last token leaves nowhere to answer from.

   Context Window Budget PlannerWindow allocation with a safety margin and the fixed overhead shown.
4. 4

   Cut the largest component first

   Usually retrieved context or conversation history. Reducing retrieved chunks from eight to four often solves it outright and improves the answer at the same time.

   RAG Retrieval Budget CalculatorWhat retrieved context costs per month, and the k comparison table.
5. 5

   Compact the history rather than truncating it

   Dropping the oldest turns loses the decisions established early, which are usually the ones that still matter. Summarising them into a compact state keeps the conclusions at a fraction of the tokens.

   Agent Memory Compaction PromptCompaction prompt that preserves decisions and numbers, drops the rest.

Stopping it happening again

• Count the assembled request in your own code before sending, and fail gracefully rather than letting the API reject it.
• Chunk documents by tokens, never by characters — character-based splitting produces chunks of wildly varying token length.
• Cap conversation history at a fixed number of turns, or compact on a threshold of roughly half the window.
• Leave headroom. Token counts for models without a public browser tokenizer are estimates, so filling a window to 99% is a decision to fail occasionally.

Frequently asked questions

Does the model’s response count towards the context window?

Yes. The window is a single budget covering input and output together. This is the part people most often miss — a document that technically fits leaves no room to answer about it.

Do tool definitions count towards the context limit?

Yes, all of them: every tool name, description and parameter schema is sent on every request, plus the provider’s own tool-use preamble of several hundred tokens.

Will a bigger context window fix this?

It will stop the error and may not fix the problem. Recall degrades as a window fills, so a tightly retrieved short prompt usually outperforms a padded long one — and costs far less, because long context is billed on every turn.

Related errors

• Response cut off mid-sentencellm response cut off incomplete max_tokens truncated
• Rate limit exceeded (429)rate limit reached 429 tokens per min