Skip to slide
11 / 29

CH 02 · How Context Goes Wrong: Rot, Pollution, and Confusion · 05/08

Read as one page

Context confusion: contradictory or indistinguishable instructions

Context confusion is the failure mode where the model cannot tell apart instructions, data, and structural markers, or where it is handed directives that contradict each other.

The article notes this frequently happens with the system instructions themselves. If the global rules clash internally, or if there are too many similar-sounding instructions, or if the system rules conflict with what the user just asked, the model gets confused about which directive to follow. A common real-world trigger, discussed more in Chapter 4, is giving the model too many tools: with 100 or more tools available, the model starts hallucinating tool parameters or calling the wrong tool entirely. That is context confusion caused by an overloaded toolset.

Finished this idea?

Your learning progress stays in this browser.