TokenPad
Reference

Streaming Latency Simulator

What streaming does to perceived speed, in numbers.

Settings
Streaming SimulatorExact
0Token change
Token change0no change
Input tokens0what you pasted
Output tokens0what you would send
Token cost of this result
Output tokens0
As input$0.00
× 100K requests$0.00

Everything on this page runs in your browser. Nothing you paste is transmitted, because there is no server here to transmit it to.

Result
 

Time to first token is where input length shows up

Before generating anything the model must process your entire prompt. On a long prompt that prefill is often the largest component of time to first token, which means cutting input improves felt speed as well as cost.

Prompt caching helps here too: a cached prefix skips most of the prefill, which is one of its less-discussed benefits.

Perceived against actual

Streaming does not make generation faster. It changes what the user waits for — from the whole response to the first token — and that is frequently a five to ten times improvement in perceived latency for no engineering cost beyond handling the stream.

The reading-speed figure tells you whether streaming will feel instant: above about 250 words per minute the text arrives faster than most people read, so the wait disappears entirely.

Frequently asked questions

Is streaming always worth implementing?
For anything a user watches, yes. For a background job nobody is watching, it adds complexity for no benefit — and those jobs usually belong on a batch endpoint anyway.
Why is my time to first token so high?
Usually prompt length. Model the prefill separately here — if it dominates, shortening the prompt improves latency more than switching to a faster model would.

More reference tools