How to Manage an AI Agent's Context Window

Quick answer
The context window is the total text a model can consider at once: instructions, conversation, file contents, and tool results. It is finite.
Older content is not remembered automatically — it stays usable only while it remains inside the window, and is dropped or summarised when it overflows.
Two identical questions cost and behave differently depending on how much content precedes them, which is why long sessions degrade.
The controls are: start fresh sessions for new tasks, keep only relevant files in context, summarise before overflow, and split large tasks into scoped ones.
Treat context as a budget you spend on the current task, not as a transcript you accumulate.

What is a context window?

It is the amount of text a model can attend to in a single request, measured in tokens — roughly three-quarters of a word each in English. Everything the model knows about your task must be inside it: the system instructions, the conversation so far, the contents of files the agent has read, and the results of commands it has run. When the total approaches the limit, the client must do something, and what it does is drop or compress older content.

That last point explains most agent forgetfulness. Nothing is retained between requests except what the client sends again. A file the agent read an hour ago is only available if it is still in the window or is re-read. A decision you made early in the session is only respected if that part of the transcript survived. Context management is the practice of making sure the important things survive and the irrelevant things do not take their place.

Step 1 — start fresh sessions for new tasks

The cheapest and most effective control is to not carry history you no longer need. Finish a task, then begin the next one in a new session, with a fresh prompt that restates what matters. A session that has accumulated an unrelated investigation is both more expensive and less accurate, because the model spends attention on text that has nothing to do with the current goal.

Carry forward only what the next step needs: the decisions you made, the files involved, and the constraint you discovered. Written down, that is usually a paragraph. Left in the transcript, it is thousands of tokens of exploration that dilute the signal.

Step 2 — keep only the files that matter in context

Agents often pull in more of the repository than the task requires, and each file consumes budget on every subsequent turn. Point the agent at the specific files involved rather than the directory, and remove or close files once the work has moved on. Where the tool supports it, exclude generated code, lock files, build output, and vendored dependencies — they are large, mostly unreadable, and rarely what the task is about.

A related habit is to give the agent a map instead of the territory. A short description of the layout and the relevant module names is far cheaper than the contents of ten files, and the agent can request the ones it needs. This is also what a good spec does: it tells the agent where to look so it does not have to search.

Context budget discipline for a coding session:
  - The task goal and its constraints            (small, always relevant)
  - The 2-5 files being changed                  (large, keep current)
  - Interfaces those files depend on             (medium, keep while touching them)
  - Test output that justifies the change         (medium, refresh, do not accumulate)
  - Failing attempt transcripts                   (large, summarise then drop)

Drop: generated code, lock files, build output, vendored deps, unrelated modules.
Start a new session when the goal changes, and restate the dropped decisions.

Step 3 — summarise before you overflow

When a session must continue, compaction — summarising older turns into a shorter form — is the tool that makes it possible. It works best when you control what survives: ask for a summary of decisions, constraints, file paths, and open questions, then start the next session from that summary rather than trusting an automatic one produced under pressure. Automatic compaction is better than losing history entirely; a deliberate summary is better than both.

In practice the pattern is a checkpoint. Every significant milestone, write down the state in a few lines: what is built, what is decided, what is next, and where the relevant code lives. That document becomes the seed for the next session and, incidentally, a useful record for a human reading the repository later.

Step 4 — scope the task to fit

Some tasks are simply too large for one session: rewrite the authentication layer, migrate the database, restructure the repository. Attempting them in one pass produces a long session with a diluted context, which is where the agent starts contradicting earlier decisions and re-editing its own work. Split along seams that can be verified independently — one module, one endpoint, one migration — and let each finish.

Scoped tasks are also cheaper and safer. A smaller surface means fewer files in context, a clearer success condition, and a diff that can be reviewed in full. The equivalent of a large pull request being hard to review is a large agent session being hard to steer, for the same reason: too much happens between two points that can be checked.

Where this bites vibecoders
The failure looks like an agent that contradicts something it agreed to twenty messages ago, or re-adds a dependency you removed. That is not a personality change, it is content falling out of the window or being compressed under load. The fix is not to argue with the agent — it is to start a fresh session seeded with the decisions that matter, or to checkpoint the state before continuing. Recognising this as a memory-limit symptom rather than a reasoning failure saves a lot of wasted correction.

Where AI coding assistants get this wrong

Continuing a long session across unrelated tasks because starting again feels like losing progress.
Leaving whole directories of generated or vendored code in context.
Trusting automatic compaction to preserve the decisions that matter.
Attempting a repository-wide refactor in one session.
Re-explaining instead of restating: feeding the transcript rather than a summary.
Assuming the agent remembers an instruction from earlier in the session once context has grown.

Checklist

Start a new session when the goal changes.
Restate decisions and constraints in the first message of a new session.
Point at specific files rather than directories.
Exclude generated code, lock files, and build output.
Write a short checkpoint at each milestone.
Split large tasks along independently verifiable seams.

FAQ

What happens when the context window fills up?

The client must drop or summarise older content, because every request has a hard token limit. In practice the earliest conversation and some file contents disappear, which is why an agent can seem to forget an instruction it followed earlier. Starting a fresh session with a restated summary is more reliable than relying on automatic compaction.

Does a larger context window fix the problem?

It postpones it. A bigger window holds more before overflow, but cost grows with context and attention is spread thinner, so quality often degrades before the limit is reached. The useful discipline is the same at any size: keep the current task’s relevant material in the window and keep everything else out.

How do I get the agent to forget a wrong approach?

Start a new session rather than correcting repeatedly in the same one, because the failed attempts remain in context and continue to influence output. Seed the new session with the correct approach and the constraints you learned, and leave the exploration behind. It is both cheaper and more effective than arguing in place.

Should I put my whole codebase in context?

No. It is expensive, it crowds out the parts that matter, and it rarely improves output because the model has to find the relevant lines among everything else. A short map of the layout plus the specific files involved gives the agent better material to work from at a fraction of the cost.