Context engineering beat prompt engineering

For two years, the craft people talked about was prompt engineering: the right phrasing, the right examples, the right persona. That craft still matters. But once a model is calling tools across many turns, the words you wrote at the top are a small fraction of what it sees. The rest is tool output, retrieved documents, earlier messages, and notes. Deciding what goes into that window, and what stays out, is now the bigger job.

Anthropic's Effective context engineering for AI agents (September 2025) puts it plainly: treat context as a finite resource and aim for the smallest set of high-signal tokens that makes the desired outcome likely. The blog post describes a set of techniques: retrieving just in time rather than loading everything up front, compacting long histories, keeping structured notes outside the window, and using sub-agents so exploratory work does not clutter the main thread.

A symptom you have probably seen

An agent performs well for the first several steps of a task, then starts repeating itself, forgetting a constraint stated at the beginning, or leaning on a document it fetched ten steps ago that is no longer relevant. Teams often describe this as the model "getting dumber." Usually the model is the same. The context got worse.

Write a context bill of materials

The most practical habit I know is to list what actually goes into the window for a representative task, with an approximate token count for each part. Here is an illustrative before-and-after for a hypothetical internal agent that answers employee questions about expense policy.

Before

After

The "after" version is not just cheaper. It is easier to read when something goes wrong, because every component has a reason to be there.

Five moves, in order of payoff

Move reference material out of the system prompt. The system prompt should state what the agent is for, what it must never do, and how to use its tools. Policies and manuals belong behind a retrieval tool, fetched when a question needs them.

Trim tool output before it reaches the model. Most APIs return far more than a task needs. A small formatting function that keeps five fields instead of fifty is one of the highest-leverage pieces of code in an agent.

Compact deliberately. When history gets long, summarize it, but specify what the summary must preserve: decisions made, constraints the user stated, open questions, and sources cited. A generic "summarize the conversation" loses exactly the details that matter.

Keep notes as artifacts. A plan file, a decision log, or a draft document that the agent reads and updates is more reliable than asking it to recall something from forty turns ago.

Isolate exploration in sub-agents. When a task requires searching broadly, let a sub-agent do the searching in its own window and return a short, sourced summary. The parent keeps a clean context for the decision.

Why this fits production reality

These techniques favor short, focused runs, which is also how most production agents work. Measuring Agents in Production reports that more than two-thirds of the deployed agents it studied run ten steps or fewer before a person steps in. Short runs with curated context are cheaper to replay, simpler to evaluate, and easier to explain to a reviewer.

The leadership angle

Context is where quality, latency, and cost meet. LangChain's State of Agent Engineering survey lists quality and latency as the top two barriers to production, and bloated context worsens both while adding to the bill. That makes context assembly worth managing explicitly:

• Treat the code that builds context as production code, with review and tests.
• Track average context size per task type, and investigate when it jumps.
• Retire "paste the whole repository into the prompt" as a debugging approach for shared agents.
• Report cost per successful task alongside quality, so the trade-off stays visible.

The teams that get this right rarely talk about clever prompts. They talk about what they removed.

Sources cited

• Anthropic, Effective context engineering for AI agents (published September 29, 2025)
• Pan et al., Measuring Agents in Production (arXiv 2512.04123, submitted December 2025)
• LangChain, State of Agent Engineering (survey of 1,340 respondents, November 18–December 2, 2025; report published February 2026)