Prompting
Chain of thought
Chain-of-thought prompting asks the model to work through intermediate reasoning before giving an answer, which improves accuracy on multi-step problems.
Also written: CoT
It works because generation is sequential: tokens the model has already produced become context for what comes next, so writing out the steps gives it somewhere to do the work.
On reasoning models this is built in and happens whether you ask for it or not, billed as output tokens you never see.
In practice
It buys accuracy on multi-step problems and pays for it in output tokens, which are the expensive side. On arithmetic, extraction with conditions and anything where the answer depends on an intermediate result, that trade is usually worth it. On classification into five known buckets, it is spending money to talk to itself.
Common questions
Does chain of thought always improve accuracy?
No. It helps on multi-step problems where the answer depends on an intermediate result. On single-step classification into known buckets it adds output tokens and latency for no measurable gain.
How much does chain of thought cost?
It multiplies output tokens, which are the expensive side — often three to ten times the tokens of a direct answer. On high-volume work that is a large recurring cost for a benefit that should be measured before it is assumed.