Context engineering: why smart models act dumb, and what good systems do about it

Select ← retrievalCompress

Context window

RelevantStaleRelevantStale

Write out → external stateIsolate → clean window

In plain terms

A model has no memory of your project, your company, or even the conversation — beyond what is physically placed in front of it at each step. That workspace is called the context window, and picturing it as a whiteboard explains almost everything.

A brilliant consultant works at your office, but with one rule: everything they know about your project must fit on a single whiteboard. Every morning, the board is what they’ve got. The skill that matters isn’t thinking harder — it’s deciding what goes on the board, what gets erased, and what gets filed in a cabinet to be pulled back when needed. Run the board badly and the consultant seems to get dumber as the project grows: important details fall off the edge, stale notes crowd out fresh ones. Nothing happened to their intelligence. The board just filled up.

The anatomy

The window is finite and, subtly, it degrades before it overflows — models pay less reliable attention as context grows crowded and messy (practitioners call the effect context rot). So serious systems make four moves, continuously. Select: retrieve only what’s relevant to this step — the three files that matter, not the whole repository. Compress: summarize what’s grown long; keep the decisions, drop the play-by-play. Write out: move durable facts — decisions made, conventions learned — into external storage (files, databases) and pull them back on demand, instead of carrying them forever. Isolate: give sub-tasks their own clean windows, so exploring one question doesn’t fill the board for every question after it.

The tempting fix — “just use a model with a bigger window” — helps less than it promises: cost scales with what you stuff in, attention still degrades, and an unmanaged big window just postpones the same failure. Anthropic’s engineering write-ups on managing context in agent systems are good primary reading here.

What good looks like

Signals a system manages context well: it summarizes its own progress as it goes, it keeps durable state outside the conversation, it starts sub-tasks clean, and its performance on hour three looks like hour one. The tell of a bad one: it gets vaguer the longer you work with it.

Where Cortex stands

Cortex handles context structurally rather than heroically: work is scoped into bounded Missions — small enough to hold coherently — and the durable facts (the plan, the definition of done, what was learned about the codebase) live outside any window as first-class records, not as chat history someone hopes survives.

Questions this paper answers

What is context engineering? The discipline of managing what a model sees at each step: selecting relevant information, compressing stale detail, storing durable facts externally, and isolating sub-tasks. It’s how long-running AI work stays coherent.

Why do AI agents lose track during long tasks? Their context window fills with accumulated detail, and attention degrades as it does. Without active selection, compression, and external storage, early decisions fall out of view and quality slides.

Isn’t a bigger context window the fix? It raises the ceiling but doesn’t remove the problem: cost grows with usage, attention still degrades in crowded windows, and unmanaged context fails at any size. Management beats capacity.