TokenPad
Datasets

Train and Validation Set Splitter

A reproducible split, so today’s evaluation is comparable to tomorrow’s.

Your input

Paste your JSONL dataset — one row per line.

449 characters10 lines0 tokensor drop a file

Dataset SplitterExact
0Training rows
Training rows0train.jsonl
Validation rows0validation.jsonl
Seed42same seed, same split — record it
Token cost of this result
Output tokens0
As input$0.00
× 100K requests$0.00

Everything on this page runs in your browser. Nothing you paste is transmitted, because there is no server here to transmit it to.

Result
 

Why the seed matters more than the split

An unseeded shuffle gives you a different split every time you run it. That means the evaluation numbers from two runs are not comparable, and a change that appears to improve the model may just be an easier validation set.

The seed here produces a deterministic shuffle: the same input and the same seed always yield the same split. Record the seed alongside your results and the comparison holds.

Choosing the validation share

Ten to twenty percent is the usual range. The consideration is absolute count rather than percentage: a validation set of thirty examples produces noisy metrics regardless of what fraction of the data it represents.

For a small dataset, take a larger share. For a very large one, a few hundred well-chosen examples measure as reliably as several thousand and cost less to run.

When not to shuffle

Shuffle by default, because data assembled chronologically or grouped by source produces a validation set that is systematically different from the training set.

Do not shuffle when order carries meaning that must not leak — most obviously time series, where training on later data and validating on earlier data tests the wrong thing entirely.

Frequently asked questions

Should I deduplicate before or after splitting?
Before, always. If the same example lands in both sets, your evaluation measures memorisation and reports it as accuracy.
Do I need a separate test set as well?
If you are going to tune hyperparameters against the validation set repeatedly, yes — otherwise you gradually over-fit to it. For a single fine-tuning run, train and validation is enough.

More datasets tools