TokenPad
Reference

o200k_base vs cl100k_base Comparator

The same text, both OpenAI encodings, exact counts.

Your input

108 characters1 lines0 tokensor drop a file

Encoding ComparatorExact
0Token change
Token change0no change
Input tokens0what you pasted
Output tokens0what you would send
Token cost of this result
Output tokens0
As input$0.00
× 100K requests$0.00

Everything on this page runs in your browser. Nothing you paste is transmitted, because there is no server here to transmit it to.

Result

Why two encodings exist

cl100k_base is the older vocabulary, used by GPT-4 and GPT-3.5. o200k_base is the current one, with roughly twice the entries, used from GPT-4o onwards.

The larger vocabulary produces fewer tokens for the same text. The gain is largest on non-English content and on code, which was a deliberate goal rather than a side effect.

Where the difference bites

A cost projection built on one encoding does not transfer to a model using another, and the error consistently flatters the older model on a per-token comparison.

The same caution applies across providers, only more so: Anthropic has stated that the tokenizer introduced with Claude 4.7 produces roughly 30% more tokens for the same input than its predecessor.

Frequently asked questions

Are both counts exact?
Yes. These are the real encoders running in your browser, not estimates. Both figures are what the respective APIs would bill.
Which one applies to my model?
o200k_base for GPT-4o and everything after it, cl100k_base for GPT-4 and GPT-3.5. The model price table records which encoding each model uses.

More reference tools