Cost
Batch API
A batch API processes requests asynchronously within a latency window of hours, in exchange for a discount that is commonly around 50%.
The model and the output are identical to a synchronous call. You are buying a discount by giving up immediacy, which makes it the only cost lever on this list with no quality trade-off at all.
Batch requests also usually sit outside the interactive rate-limit quota, so moving work there frees throughput as well as money.
In practice
Batch pricing is typically half, in exchange for a completion window measured in hours. Evaluation runs, overnight classification of a backlog, re-summarising yesterday's documents and any backfill all qualify. The common mistake is not knowing which of your workloads are genuinely synchronous — most teams find that a third of their spend never needed an immediate answer.
Common questions
What workloads should go through a batch API?
Anything that does not need an answer within the hour: evaluation runs, backfills, overnight classification of a queue, re-summarising yesterday's documents. Most teams find a third of their spend never needed an immediate response.
How much cheaper is batch processing?
Typically half the synchronous rate. The trade is a completion window measured in hours rather than seconds, with no guarantee about where in that window your job finishes.