RAG
Chunking
Chunking splits documents into passages small enough to embed usefully and large enough to be interpretable on their own.
For most prose corpora the useful range is 200 to 500 tokens with ten to twenty percent overlap. Too small and a chunk loses the context that makes it meaningful; too large and one relevant sentence is diluted among many irrelevant ones.
Split on sentence or paragraph boundaries rather than at exact token indices. A chunk beginning mid-clause embeds badly and reads badly when it is later shown back to the model.
In practice
A chunk that starts mid-sentence embeds as a fragment nobody would have written, so it matches badly, and it reads badly when it is shown back to the model later. Break at paragraphs and sections, never at a fixed character count, and keep tables and code blocks whole — half a table looks like data and is missing the rows that would have contradicted the answer.
Common questions
What is the best chunk size?
The one your documents support, found by testing retrieval quality. Dense reference material wants small chunks, narrative wants larger ones, and the default in the first tutorial you read was chosen for neither.
Should I chunk by characters or by tokens?
Neither, primarily. Break at semantic boundaries — paragraphs, sections, list items — and use a token ceiling to decide when to start a new chunk. A chunk that starts mid-sentence embeds and reads badly.
Measure it
- Text Splitter for RAG and EmbeddingsToken-accurate chunking with configurable overlap. Respects sentence and paragraph boundaries.
- Chunk Overlap VisualizerShows the repeated text between chunks and the storage it costs.
- RAG Chunk Quality CheckerFlags chunks that start mid-clause, refer to nothing, or are pure pointers.