Context Engineering vs Prompt Engineering: What Actually Changed (2026)

If you’ve built anything with LLMs in the last year, you’ve watched the vocabulary shift. Job posts stopped saying “prompt engineer.” Blog posts started declaring prompt engineering dead. The replacement everyone landed on is context engineering.

Most of the takes are overheated. But the underlying change is real, and it matters if you’re shipping agents rather than demos. Here’s what actually changed, and what it means for how you build.

What is prompt engineering?

Prompt engineering is optimizing the instruction you send to a model for a single call. You control the wording, the role, the output format, the examples, and the reasoning style you ask for.

It works, and it still works. Give a model a vague instruction and you get vague output. Give it a precise role, a clear task, two good examples and an output schema, and quality jumps — no architectural change required.

Prompt engineering’s assumption is that the model call is the unit of work. You send a prompt, you get an answer, you’re done.

What is context engineering?

Context engineering is deciding what occupies the model’s context window at every step of a running system. Andrej Karpathy described it as the art and science of filling the context window with just the right information for the next step.

The context window isn’t just your prompt. At inference time it holds:

• the system prompt
• tool definitions — every tool’s name, description and parameter schema
• retrieved documents from RAG or search
• message history — every prior turn, including tool calls and their results
• memory — facts carried across sessions
• the current user input

All of it competes for the same finite budget. Context engineering is the discipline of curating that set: what to include, what to fetch on demand, what to summarize, what to drop, and what to push out to storage.

The shift in assumption is the whole story: the agent loop is the unit of work, not the single call.

Context engineering vs prompt engineering: the difference in one table

	Prompt engineering	Context engineering
Unit of work	One model call	A whole agent run (10s–100s of calls)
What you control	Instruction wording, format, examples	Everything in the window: prompt, tools, retrieval, history, memory
Main failure mode	Model misunderstands the ask	Model has wrong, stale, missing or too much information
Optimizes for	Response quality	Signal-to-noise ratio under a token budget
Typical fix	Rewrite the prompt	Change retrieval, compact history, cut tools, offload state
Cost impact	Minor	Major — context size drives cost and latency per turn
Still needed in 2026?	Yes — it’s a subset	Yes — it’s the containing discipline

The honest summary: prompt engineering didn’t die, it got absorbed. A great system prompt is now one component of a well-engineered context, and it’s still the component people get wrong most often.

Why prompting alone stopped being enough

Three things broke the one-prompt model.

1. Agents run in loops. A single-turn chatbot has a clean window every time. An agent that searches, reads files, calls APIs and iterates accumulates history at every step. By turn forty, the window is mostly tool output — and none of it was written by you.

2. Bigger windows made accuracy worse, not better. This is the counterintuitive one. Chroma’s 2025 study across 18 frontier models found accuracy degrading at every increment of context growth, even when all the relevant information stayed present. The lost-in-the-middle effect is the sharpest version: information buried in the middle of a long context gets recalled far less reliably than the same information at the start or end.

The industry calls this context rot. It means a million-token window is a budget, not a guarantee. Filling it costs you accuracy, latency and money at the same time.

3. Context became the cost center. Every token in the window is billed on every turn. An agent carrying 80k tokens of accumulated history pays for those 80k tokens forty times over a single session. Trimming context isn’t hygiene — it’s the main lever on unit economics.

The techniques that actually work

Anthropic’s engineering guidance frames the goal well: find the smallest set of high-signal tokens that maximizes the likelihood of your desired outcome. Four techniques do most of the work.

Just-in-time retrieval

Don’t preload everything the agent might need. Give it the ability to fetch, and let it fetch when the task demands it. Loading twenty documents “just in case” burns budget and adds distractors; retrieving the two that matter mid-task keeps the window clean.

This is where a well-built RAG pipeline earns its keep — not as a way to stuff documents into a prompt, but as the agent’s on-demand memory.

Compaction

When history grows past a threshold, summarize the old turns and continue from the summary. You keep conversational continuity and reclaim most of the space. Claude Code’s /compact is the visible version of this, but any long-running agent needs an equivalent.

The trade-off is real: compaction is lossy. Whatever the summarizer drops is gone. Which is why serious agents pair it with the next technique.

External note-taking

Have the agent write structured state to a file, database or scratchpad outside the window — a task list, findings so far, decisions made. The window stays small; the state persists indefinitely and survives compaction.

This is the closest thing to a free lunch in context engineering. Storage is effectively free; context is not.

Sub-agents

Delegate a bounded subtask to a fresh agent with its own clean window, and return only the summary to the parent. The parent’s context stays small while the work still gets done, and parallel sub-agents get you throughput on top.

Anthropic reported that a multi-agent setup — an Opus 4 lead delegating to Sonnet 4 sub-agents — outperformed a single Opus 4 agent by 90.2% on their internal research eval. The gain isn’t magic: it’s four clean context windows instead of one polluted one.

The strongest long-horizon agents combine all three of compaction, note-taking and sub-agents — compaction for conversation length, notes for durable state, sub-agents for focused subtasks.

Underneath it all: tool design

Every tool definition sits in the window on every call. Ten tools with verbose descriptions and sprawling schemas is a permanent tax before the agent does anything.

Two rules that consistently pay off:

• Fewer tools, clearly distinct. Overlapping tools produce wrong selections and burn tokens describing the distinction.
• Return only what’s needed. A tool that dumps a full API response into the window is a context leak. Filter server-side, return the three fields the agent uses.

So which one should you learn?

Both — in that order.

Learn prompt engineering first. It’s the faster feedback loop and it’s still where most quality problems live. If your system prompt is vague, no amount of retrieval architecture rescues it.

Learn context engineering when your system grows a loop. The trigger is concrete: the moment your app calls tools, runs multiple steps, remembers across sessions, or answers over your own documents, window management becomes the thing that decides whether it works.

For a straight single-turn classifier or a Q&A bot with no tools, context engineering is over-engineering. Don’t build an agent architecture for a problem that a good prompt solves.

A practical checklist

Before you ship an agent, check:

1. Do you know your token budget per turn? If not, measure it — most teams are surprised.
2. Is anything preloaded that could be retrieved on demand? Move it.
3. What happens at turn 50? If the answer is “we hit the window limit,” you need compaction.
4. Where does state live? If the only copy is in the message history, you’ll lose it.
5. How many tools are defined, and are any of them ambiguous? Cut and sharpen.
6. What does a tool return? If it’s a raw API payload, filter it.
7. Could a subtask run in its own window? That’s a sub-agent.

The bottom line

Prompt engineering is not dead — it’s a subset of a bigger discipline that grew around it. The real change is that context is now the scarce resource in LLM applications, and the teams shipping reliable agents are the ones treating it that way: measuring it, budgeting it, and defending it.

The best prompt in the world can’t help a model that’s looking at the wrong information.

---

I build production RAG systems, AI agents and voice agents — the kind where context budgets decide whether the thing works at 3am. If you’re wrestling with an agent that degrades over long sessions, see what I work on or look at the projects.