TokenPad
Optimisation

Data Format Token Comparison

The same data as JSON, CSV, YAML or a markdown table. One is much cheaper.

Your data (JSON)
RecoverableExact
0%vs the worst format
Cheapest—
Tokens saved—

CSV wins on flat tabular data because it names each field once instead of once per record. On nested or irregular data the ordering changes, which is why this measures your data rather than repeating a rule.

Same data, every format

FormatTokensvs cheapestCost / 1K requestsShare
0—$0.00
0—$0.00
0—$0.00
0—$0.00
0—$0.00
0—$0.00

CSV

id,name,plan,seats,status
1041,Ana Ruiz,pro,12,active
1042,Luis Peña,team,40,active
1043,Mia Ortiz,free,1,churned

What this tool tells you

The same records, serialised six ways, with the token cost of each. On flat tabular data the spread between the cheapest and the most expensive format is routinely three to one — and almost everyone is sending the most expensive one, because it is what JSON.stringify(data, null, 2) produces.

Why the spread is so large

Three costs stack up, and only one of them is the data:

  • Indentation. Two to four tokens per line, scaling with nesting depth, carrying no information.
  • Punctuation. Braces, brackets, colons, commas and quotes each take a token.
  • Repeated keys. The big one. An array of 500 records with six fields repeats those six key strings 500 times — 3,000 repetitions conveying six keys of information.

CSV eliminates all three: field names appear once in a header row, there is no indentation, and the punctuation is a single comma per field. That is the whole reason it usually wins.

Choosing between them

CSV

Cheapest for flat tabular data by a wide margin, and models parse it reliably provided the header row is present and adjacent to the rows. The failure mode is quoting: a field containing a comma or a newline shifts a column if it is not escaped properly. Test with your messiest real record, never a clean sample.

TSV

Marginally different from CSV in token terms and safer when fields commonly contain commas. Worth checking against your data rather than assuming.

JSON with columns and rows

Keeps JSON's explicit structure while stating each key once. A good middle ground when downstream code expects JSON but you want the tabular saving.

Markdown table

More expensive than CSV because of the pipes and the separator row, and worth it only when the same content is also being shown to a person.

Minified JSON

The right answer for nested or irregular data, where the tabular formats do not apply at all. Still typically 30–50% cheaper than the indented version.

Where a cheaper format is the wrong call

Nesting, irregular shapes and explicit types are what JSON is buying you. If records have different fields, contain nested objects, or rely on the distinction between the number 0 and the string "0", flattening to CSV loses information the model may need — and the tool will tell you the tabular formats are unavailable rather than producing something misleading.

The general principle: format changes are prompt changes. Re-run your evaluations, exactly as you would after editing an instruction.

Stacking the savings

Format choice and whitespace removal compound. Pick the cheapest viable format here, then run it through the prompt optimizer for anything left over. Measure the winner against every model in the token counter, since the saving is not identical across tokenizers.

For the reasoning behind all of this in one place, see why your JSON costs three times more tokens than you think.

Frequently asked questions

Which format is cheapest?
For tabular data, CSV almost always, because it states each field name once instead of once per record. Pretty-printed JSON is almost always the most expensive. For deeply nested or irregular data the ordering changes, which is why the tool measures your actual data rather than quoting a rule.
Do models handle CSV as well as JSON?
For flat tabular data, yes, provided the header row is present and adjacent to the rows. Models parse delimited data reliably. Where JSON earns its cost is nesting, irregular shapes and explicit types — if your data has those, the saving is smaller and the risk is higher.
Is this safe to do in production?
Change the format and re-run your evaluations, as with any prompt change. The failure mode to watch for is ambiguity, not comprehension: a CSV field containing a comma or a newline can shift a column if it is not quoted properly. Test with your messiest real record, not a clean one.

More optimisation tools