What this tool tells you
Retrieval-augmented generation lives or dies on how the source material was divided. This splits text into chunks measured in real tokens rather than characters, with configurable overlap, and without cutting through the middle of a sentence.
Every chunk shows its exact token count, because the limit your embedding model enforces is denominated in tokens and a character-based splitter produces chunks of wildly varying token length. Some of those will quietly exceed the limit and be truncated, and truncation in an embedding pipeline is silent — the vector is simply wrong and nothing tells you.
Choosing a chunk size
For most retrieval work, 200 to 500 tokens. The trade-off is precision against interpretability:
- Too small and a chunk loses the context that makes it meaningful. A sentence referring to "the second approach" is useless without the paragraph that introduced the first. Retrieval succeeds and the answer is still wrong.
- Too large and the embedding dilutes. One relevant sentence among two hundred irrelevant ones produces a vector that sits nowhere near the query. Retrieval simply misses.
Start at 300 and let the failure mode tell you which way to move. If the right passage is never retrieved, go smaller. If it is retrieved but the answer is incoherent, go larger.
Why overlap is not optional
Meaning does not respect chunk boundaries. A sentence split across two chunks appears complete in neither, so neither embedding represents it and it becomes effectively unsearchable. Ten to twenty percent overlap means any given sentence appears whole in at least one chunk.
The cost is duplicated storage and a slightly larger index. Compared to a retrieval failure that silently returns the wrong passage, it is cheap. The readout above shows the duplication explicitly, so you can see what you are trading.
Sentence, paragraph or exact token
Sentence is the sensible default. Chunks stay close to the target size and never begin mid-clause.
Paragraph suits documentation and articles where a paragraph is a self-contained idea. Chunk sizes vary more, and a single long paragraph can exceed the ceiling — the tool flags that rather than silently splitting it.
Exact token produces perfectly uniform chunks and cuts wherever it lands, including mid-word. Useful when you need deterministic sizing for a fixed-shape index, and a poor default for anything a model will read back.
What happens after chunking
Chunks get embedded, stored, and later retrieved into a prompt — which brings back the constraint everything else here is about. If you retrieve eight chunks of 400 tokens, that is 3,200 tokens of context on every single request, billed every time. Check what your retrieved set costs in the cost calculator, and confirm it fits alongside your system prompt in the context window calculator.
Retrieving fewer, better chunks beats retrieving more, and it is cheaper on both the embedding and the generation side. That is the whole argument for spending time on chunk quality rather than raising the retrieval count until something works.
A note on the counts
Chunking here uses o200k_base, the current OpenAI encoding, so the counts are exact for OpenAI models and close for most others. Embedding models from other providers tokenize differently — leave a margin below their stated limit rather than filling it exactly. If you need the source document measured against a specific model first, use the token counter.