Skip to slide
046 / 170

CH 04 · Tokens, Context Windows, and the Quadratic Problem · 11/13

Read as one page

Read the source yourself

Five stops, in the order the chapter used them. Each link is pinned to the commit these slides were read against.

  1. LiteLLM's model registry. Read sample_spec at the top of the file first, then the claude-sonnet-4-5 entry. Check which field is the window and which field the template calls legacy.
  2. OpenCode's Token.estimate, then outputLimit. outputLimit is where the guess changes maxTokens. Compaction also uses Token.estimate, in oldestToDrop in compaction.ts, to choose which older messages to drop, so the guess changes the prompt itself.
  3. pi's estimate.ts. Read estimateMessageTokens, then estimateContextTokens further down that file. The second function is where a real provider total replaces the guess.
  4. pi's calculateCost, then the first case in model-cost-tiers.test.ts. Base input is $0.10 per million tokens. The step starts once input is over 100,000 tokens. At 100,000 the rate is still $0.10 per million. At 150,000 fresh input tokens, calculateCost bills all 150,000 at $0.50 per million, which is $0.075.
  5. Codex's for_prompt, then normalize_history in the same file. Then read get_incremental_items and list the cases that send the full input anyway.

Try this on the toy. Change estimate_tokens so it rounds up the way Codex does: return max(1, (len(text) + CHARS_PER_TOKEN - 1) // CHARS_PER_TOKEN). The printout does not call estimate_tokens. Compare estimate_tokens("abcde") and estimate_tokens("abcdef") before and after the edit: 5 characters go from 1 to 2, and 6 from 1 to 2. The session printout stays put until simulate_session uses that function.

Finished this idea?

Your learning progress stays in this browser.