Context Window

The context window is the total amount of text a language model can hold in view at once. It is measured in tokens, and it covers everything: your instructions, any documents you paste in, the conversation so far, and the reply being written.

It is a working desk, not a memory. Everything the model can use has to fit on the desk right now. When the conversation ends, the desk is cleared.

Think of it like this. Think of someone reading with a single page visible at a time. They can reason brilliantly about what is on that page, and they cannot see anything that fell off it. Adding more pages means a bigger page, not a better memory.

How it works

• Counted in tokens, not characters or words.
• Everything shares the budget. System instructions, retrieved documents, conversation history, and the output all come out of the same total.
• When it fills, something goes. Older messages get dropped or summarised, which is why long chats start forgetting the beginning.
• Bigger windows cost more. Memory and compute grow sharply with length, which is why long context is priced higher.

Common mistakes

• "It is the model's memory." It resets completely between sessions. Nothing persists unless you store it yourself.
• "A bigger window always gives better answers." Accuracy can drop on very long inputs as attention spreads thin. Relevant beats plentiful.
• "Only my input counts." The reply is drawn from the same budget, so a full window leaves no room to answer.
• "Context window and training data are the same thing." Training is what the model learned before you arrived. Context is what you hand it now.