Skip to slide
049 / 221

CH 04 · Model-Provider Abstraction and Multi-Model Strategy · 04/15

Read as one page

Real code: pi rewrites a turn from another model

Those disagreements sit in the history, so a turn from another provider, api, or model id cannot be sent on unchanged. pi repairs that in transformMessages (packages/ai). The Anthropic, Google, OpenAI, Bedrock, and Mistral adapters each call it before building a request. An AssistantMessage is one assistant turn: provider, api, model id, and content blocks. The api is the wire protocol, such as anthropic-messages. model is the model about to be called.

if (msg.role === "assistant") {
  const assistantMsg = msg as AssistantMessage;
  const isSameModel =  // [1]
    assistantMsg.provider === model.provider &&
    assistantMsg.api === model.api &&
    assistantMsg.model === model.id;

  const transformedContent = assistantMsg.content.flatMap((block) => {
    if (block.type === "thinking") {
      // ... (3 lines: why thinking from another model is dropped)
      if (!isSameModel) return [];  // [2]
      // Keep redacted and signed blocks even if the thinking text is empty
      // (OpenAI encrypted reasoning); they are needed for replay.
      if (block.redacted || block.thinkingSignature) return block;  // [3]
      // ... (2 lines: drop unsigned thinking whose text is empty)
    }
    // ... (7 lines: text from another model keeps the words and drops textSignature)
    if (block.type === "toolCall") {
      const toolCall = block as ToolCall;
      let normalizedToolCall: ToolCall = toolCall;

      if (!isSameModel && toolCall.thoughtSignature) {  // [4]
        normalizedToolCall = { ...toolCall };
        delete (normalizedToolCall as { thoughtSignature?: string }).thoughtSignature;
      }

      if (!isSameModel && normalizeToolCallId) {  // [5]
        const normalizedId = normalizeToolCallId(toolCall.id, model, assistantMsg);
        if (normalizedId !== toolCall.id) {
          toolCallIdMap.set(toolCall.id, normalizedId);
          normalizedToolCall = { ...normalizedToolCall, id: normalizedId };
        }
      }

      return normalizedToolCall;

Source: earendil-works/pi, packages/ai/src/api/transform-messages.ts (MIT)

What to notice

  1. [1] isSameModel is true only when provider, api, and model id all match. Checking the id alone would replay a turn from a different provider, or from a different api on the same provider.
  2. [2] A thinking block is the model's reasoning, stored apart from the reply. When the provider, api, or model id differs, transformMessages removes that block. The cut comment says the signature is valid only for the model that produced it, and that readable thinking sent as ordinary text would be treated as part of the reply.
  3. [3] On the same model, a block is kept when it is redacted or carries a thinkingSignature, even when the thinking text is empty. thinkingSignature is opaque replay data, and redacted means safety filters removed the text and left an encrypted payload in that signature for the next request. The comment on the kept line names OpenAI encrypted reasoning. The cut lines drop empty unsigned thinking, and they drop textSignature from another model's text.
  4. [4] thoughtSignature sits on the tool call. The ToolCall type calls it Google-specific: an opaque signature for reusing thought context. When the model changed, transformMessages copies the call and deletes the field, because leaving the signature in place would attach one model's thought context to another model's request.
  5. [5] normalizeToolCallId runs only when the model changed, so a same-model conversation keeps its ids. The comment above transformMessages says OpenAI Responses ids run to 450 characters or more and can contain |, while Anthropic requires ^[a-zA-Z0-9_-]+$ and at most 64 characters. Anthropic's function replaces each character outside letters, digits, _, and - with _, then slices to 64. The new id goes into toolCallIdMap, and a later tool result rewrites its toolCallId from that map.

Read it yourself: Read closePendingToolCalls. It inserts an error tool result, text No result provided, when a call has no result. Then apply normalizeToolCallId on paper to | plus 80 a characters. You should get _ plus 63 a characters.

Finished this idea?

Your learning progress stays in this browser.