Expert knowledge for digital decisions

What is the context window of a language model?

Short answer

The context window is the amount of text a model can consider at once when processing a query – including the question, instructions, documents, and previous conversation history. When it is full, older content is discarded. A large window does not mean it should be filled completely: the more text, the more likely the model will overlook essential details.

What is included

Everything the model sees during a query:

The system instruction
Attached document excerpts
The previous conversation history
The current question
And the answer it generates

What happens when it is full

Depending on the implementation, the beginning is cut off or the query is rejected. In long conversations, this means the model "forgets" what was discussed earlier – without indicating this.

A large window is not automatically better

Two reasons:

Costs. Everything in the window is charged, repeatedly for every query.

Quality. With very long inputs, the reliability of a model finding a specific piece of information decreases – especially if it is in the middle.

A carefully selected two-page excerpt delivers better results than a complete manual.

What follows

Not "throw everything at the model, it will handle it," but deliberately curate: find the relevant sections and provide only those. This is exactly the principle of RAG.

Practical note

When an assistant starts contradicting itself after a long conversation, the context window is usually the cause – not the model.

Key facts

Contains: Instruction, Documents, History, Question, Answer
Large is not better: Higher costs, reduced accuracy

What we offer on this topic

We start with a clearly defined use case and integrate AI into your existing processes.

AI automation

Sources

All external claims are backed by traceable sources.

01
KI-Lösungen in der Cloud und auf eigenen Servern Ouhud GmbH