CH 14 · Cost, Latency, and Model Tiering · 05/14
Read as one pageTurn reasoning on and off per call
Reasoning is a per-call decision (Chapters 4, 8). On for hard interactive tasks where it improves quality and the user benefits from seeing it; off for bulk and one-shot jobs, where you should also explicitly zero the thinking budget where the provider allows, so you actually stop paying for it. A bulk extraction over a thousand documents with reasoning left on can multiply the bill for no benefit.
Finished this idea?
Your learning progress stays in this browser.