CH 04 · Tokens, Context Windows, and the Quadratic Problem · 11/13
Read as one pageRead the source yourself
Five stops, in the order the chapter used them. Each link is pinned to the commit these slides were read against.
- LiteLLM's model registry. Read
sample_specat the top of the file first, then theclaude-sonnet-4-5entry. Check which field is the window and which field the template calls legacy. - OpenCode's
Token.estimate, thenoutputLimit.outputLimitis where the guess changesmaxTokens. Compaction also usesToken.estimate, inoldestToDropincompaction.ts, to choose which older messages to drop, so the guess changes the prompt itself. - pi's
estimate.ts. ReadestimateMessageTokens, thenestimateContextTokensfurther down that file. The second function is where a real provider total replaces the guess. - pi's
calculateCost, then the first case inmodel-cost-tiers.test.ts. Base input is $0.10 per million tokens. The step starts once input is over 100,000 tokens. At 100,000 the rate is still $0.10 per million. At 150,000 fresh input tokens,calculateCostbills all 150,000 at $0.50 per million, which is $0.075. - Codex's
for_prompt, thennormalize_historyin the same file. Then readget_incremental_itemsand list the cases that send the full input anyway.
Try this on the toy. Change estimate_tokens so it rounds up the way Codex does: return max(1, (len(text) + CHARS_PER_TOKEN - 1) // CHARS_PER_TOKEN). The printout does not call estimate_tokens. Compare estimate_tokens("abcde") and estimate_tokens("abcdef") before and after the edit: 5 characters go from 1 to 2, and 6 from 1 to 2. The session printout stays put until simulate_session uses that function.
Finished this idea?
Your learning progress stays in this browser.