Definition
Context Window
A context window is the maximum amount of text — measured in tokens — that a language model can consider at once, spanning its input and its output. Everything the model 'sees' for a given request must fit inside it: the prompt, retrieved data, conversation history, and the answer. When content exceeds the window, the oldest or least relevant parts must be dropped.
Key takeaways
- A context window is the maximum amount of text — measured in tokens — that a language model can consider at once, spanning its input and its output. Everything the model 'sees' for a given request must fit inside it: the prompt, retrieved data, conversation history, and the answer. When content exceeds the window, the oldest or least relevant parts must be dropped.
- Tokens are the unit a model reads in — roughly chunks of words, where a token averages about four characters of English.
- Context windows have grown dramatically, but bigger is not free.
Tokens are the unit a model reads in — roughly chunks of words, where a token averages about four characters of English. The context window is the hard ceiling on how many tokens a single request may use across both what you send and what the model generates. A model with a large window can reason over a long document or a deep conversation in one shot; a small window forces the surrounding system to be selective about what it includes.
This ceiling is the central constraint of building with LLMs. You cannot simply pour an entire knowledge base into a prompt — it won't fit, and even within the limit, models attend less reliably to information buried in the middle of a very long context. The engineering task becomes choosing what to include: retrieving only the most relevant passages, summarizing history, and ordering content so the important parts are where the model uses them best.
Context windows have grown dramatically, but bigger is not free. Larger contexts cost more to process and can dilute focus, so the goal is rarely to fill the window but to fill it well. This is exactly why semantic retrieval matters — it selects the small, relevant slice of data worth spending context on, rather than everything that might conceivably help.
Related terms
- Large Language Model (LLM)A large language model is an AI system trained on vast amounts of text to predict and generate language, enabling it to answer questions, summarize, write, and reason over natural language. LLMs power modern AI assistants and agents. They are probabilistic next-token predictors, which makes them remarkably capable but also prone to confident errors.
- Retrieval-Augmented Generation (RAG)Retrieval-augmented generation (RAG) grounds a language model's answers in your own data by retrieving the most relevant documents at query time and feeding them into the model's context before it responds. Instead of relying solely on what the model memorized in training, RAG lets it answer from current, authoritative, private sources — sharply reducing hallucination.
- Semantic SearchSemantic search finds results by meaning rather than exact keywords, using vector embeddings that place similar concepts near each other in mathematical space. A query for 'login broken' can surface an issue titled 'users can't authenticate' even with no shared words. It powers more relevant search and is the retrieval layer behind many AI features.
- AI AgentAn AI agent is a software system that uses a large language model to pursue a goal across multiple steps — reading context, choosing tools, and taking actions — rather than answering a single prompt. In a work platform, agents triage issues, draft updates, and execute multi-step tasks as autonomous teammates, bounded by the permissions and approvals their operators set.