CH 04 · Tokens, Context Windows, and the Quadratic Problem · 09/13
Read as one pageReal code: pi prices the provider's real usage
CostModel.turn_cost multiplies your own guess by arbitrary rates per thousand tokens. pi waits until the provider answers, then prices the counts the provider reports. Those counts sit on a Usage value attached to the assistant message.
export interface Usage {
input: number; // [1]
output: number;
cacheRead: number;
cacheWrite: number;
/** Subset of `cacheWrite` written with 1h retention. Only Anthropic reports this split. */
cacheWrite1h?: number; // [2]
// ... (reasoning tokens are a subset of output, so they are already counted)
reasoning?: number;
totalTokens: number;
cost: { // [3]
input: number;
output: number;
cacheRead: number;
cacheWrite: number;
total: number;
};
}
Source: earendil-works/pi, packages/ai/src/types.ts (MIT)
calculateCost fills usage.cost from the model's price table. The rates are US dollars per million tokens, the same kind of numbers as the LiteLLM entry earlier, written per million instead of per token.
export function calculateCost(model: AnyModel, usage: Usage): Usage["cost"] {
const inputTokens = usage.input + usage.cacheRead + usage.cacheWrite; // [4]
let rates: ModelCostRates = model.cost;
let matchedThreshold = -1;
for (const tier of model.cost.tiers ?? []) { // [5]
if (inputTokens > tier.inputTokensAbove && tier.inputTokensAbove > matchedThreshold) {
rates = tier;
matchedThreshold = tier.inputTokensAbove;
}
}
// Anthropic charges 2x base input for 1h cache writes.
const longWrite = usage.cacheWrite1h ?? 0; // [6]
const shortWrite = usage.cacheWrite - longWrite;
usage.cost.input = (rates.input / 1000000) * usage.input; // [7]
usage.cost.output = (rates.output / 1000000) * usage.output;
usage.cost.cacheRead = (rates.cacheRead / 1000000) * usage.cacheRead;
usage.cost.cacheWrite = (rates.cacheWrite * shortWrite + rates.input * 2 * longWrite) / 1000000;
usage.cost.total = usage.cost.input + usage.cost.output + usage.cost.cacheRead + usage.cost.cacheWrite;
return usage.cost;
}
Source: earendil-works/pi, packages/ai/src/models.ts (MIT)
What to notice
- [1]
inputis fresh tokens,outputis tokens written,cacheReadis a cache hit,cacheWriteis a cache fill. Yourturn_costhas no write price, so a cache fill looks free. - [2]
cacheWrite1his the part ofcacheWritestored for one hour. Anthropic reports that split.reasoningis already insideoutput, so leave it out of the sum. - [3]
costis written here, in dollars.totalTokensis the provider's own total when it sends one. - [4] The tier check adds all three input kinds. A cached token still counts toward how big the prompt was.
- [5]
inputTokensAboveis where a higher price starts. The highest step the prompt has passed sets the rates for the whole request. The price schema calls that request-wide. - [6] One-hour writes are billed at twice the base input rate.
?? 0treats a missing split as none. Your singlecached_ratecannot say this. - [7] Dividing by one million turns a per-million rate into a per-token rate. Your toy divides by one thousand, and its rates are arbitrary units.
The Anthropic adapter fills these fields from message_start, adds totalTokens itself (its comment says Anthropic provides none), then calls calculateCost.
Read it yourself: open anthropic-messages.ts at message_start and write down which provider field lands in which Usage field. Then search for message_delta in the same function, about 130 lines later. That branch writes a field only when the event includes it. A missing input_tokens stored as 0 would wipe the count from message_start.
Finished this idea?
Your learning progress stays in this browser.