The problem: long inputs break models

The context window is the model's working memory. Two things go wrong when inputs get large:

First, the hard limit: if the text simply does not fit in the window, the model cannot see all of it. Second, and more subtly, even when text does fit, models tend to degrade as the input grows longer. Buried details get overlooked, and the model effectively forgets parts of what it read, as if some early information had to be pushed aside to make room. So just building ever-bigger context windows is not a full solution; quality slips as length climbs.

flowchart LR
    Short[Short input] --> Good[Strong, accurate answers]
    Long[Very long input<br/>stuffed into the window] --> Bad[Details missed,<br/>accuracy drops]