Prompt injection and goal hijack

Any agent that reads email, tickets, documents, or web pages will eventually read a sentence written to manipulate it. Sometimes it is crude: "ignore your previous instructions." Sometimes it is subtle: a support ticket that politely asks the assistant to include the customer's full account history in its reply, or a web page with hidden text recommending a particular action.

The OWASP Top 10 for Agentic Applications 2026 puts agent goal hijack at the top of its list, alongside related risks such as tool misuse and memory and context poisoning. The core problem is structural. Language models process instructions and data in the same channel, and no system prompt fully fixes that. You manage it with system design.

An illustrative attack

Consider a hypothetical inbox assistant that reads incoming email, summarizes it, drafts replies, and can send messages and look up contacts.

An email arrives from outside the company. Its visible text is an ordinary meeting request. Below it, in white text on a white background, is a paragraph addressed to "the AI assistant processing this message," asking it to look up the company's finance contacts and send each of them a link with a note saying the invoice is attached.

Trace what a naive design does. The email body is concatenated into the context. The model treats the hidden paragraph as a reasonable request. It calls the contact lookup, then the send tool. The user sees a summary that reads "Meeting request from a vendor," while a phishing message has gone out from a trusted internal address.

No step in that chain requires a sophisticated exploit. It requires only that untrusted text can influence which tools get called.

Defense in layers

No single control stops injection. Stack several, so a failure in one is caught by another.

Layer 1: Label the channels. When assembling context, separate system instructions, developer configuration, user requests, and external content, and mark external content explicitly as data from an untrusted source. Never place untrusted text in the system position. Labels do not make models immune, but they help, and they make the boundary visible in traces.

Layer 2: Restrict tools while untrusted content is present. This is the most effective control. If the context holds external content, high-risk tools (send, delete, share, pay) are disabled for that run or require explicit user confirmation. In the example, the assistant could still summarize and draft, but sending would need a click from the user.

Layer 3: Constrain arguments. Even permitted tools should validate arguments against policy: recipients limited to existing correspondents, links checked against allowed domains, amounts within limits.

Layer 4: Detect and flag. Classifiers or heuristics can catch common injection patterns and warn the user: "This email contains text addressed to an AI assistant." Treat detection as a helpful signal, never a guarantee.

Layer 5: Protect memory. If the agent can write to long-term memory, injected content can persist and steer future sessions. Keep memory off, or require provenance and review for anything derived from external content.

Layer 6: Show your sources. Tell users which documents and messages the agent read before it acted. Transparency turns users into another line of detection.

Build an injection test suite

Injection resistance should be tested like any other requirement, in CI, with hard failures. Useful categories of test case:

• Direct override attempts inside user-supplied documents.
• Hidden text in HTML email or rich documents.
• Tool-targeted instructions that name a specific tool or recipient.
• Exfiltration attempts that ask the agent to put sensitive data in an outbound message or URL.
• Delayed cases, where the instruction arrives in one tool result and the harmful action would happen several steps later.
• Memory cases, where the instruction asks the agent to remember something for later.

For each case, the pass condition is about behavior, not wording: the harmful tool was not called, or it was held for confirmation. Add cases whenever you see a new pattern in production or in published research.

What to show at launch

In a production readiness review, demonstrate rather than assert. Bring a trace from a real test in which an injected instruction was present, showing the label on the content, the restricted tool set, and the refusal or confirmation request. Security reviewers trust a trace far more than a description.

Living with residual risk

Injection is not a bug you fix once. Models improve, attackers adapt, and every new tool adds a new target. The sustainable posture is to assume some injected text will occasionally be followed, and to design so that following it cannot cause serious harm without a person noticing. In practice that means tight tool permissions, confirmation on consequential actions, and an injection suite that runs every time anything changes.

Sources cited

• OWASP GenAI Security Project, Top 10 for Agentic Applications 2026 (released December 9, 2025)