Cost
Input tokens
Input tokens are everything you send to the model: system prompt, conversation history, tool definitions, documents and the user message.
Also written: prompt tokens
They are processed in a single parallel forward pass, which is why they are priced well below output tokens — commonly a quarter to a sixth of the output rate.
The number that matters is the whole request, not the user’s message. In a retrieval-augmented system the retrieved passages usually dominate everything else; in an agent, the tool definitions do.
In practice
Input is the cheap side and the side that grows without anyone deciding to grow it. A system prompt that gained a line a month for two years is 24 lines of tax on every request forever. At a million requests a month, 200 extra input tokens on a $3/1M model is $600 a month for text nobody has read since it was added.
Common questions
How do I reduce input tokens without losing quality?
Read the system prompt line by line and delete what stopped mattering, retrieve from long documents instead of pasting them, and trim conversation history rather than resending it whole. Those three cover most of the available reduction.
Are input tokens cheaper than output tokens?
Substantially — typically three to five times cheaper on the same model. That inverts the common intuition that long prompts are the expensive part; a verbose answer usually costs more than a verbose question.