AI Workflows · Guide
Keep Retrieved Documents From Hijacking an AI Workflow
Treat external pages and uploaded files as task data even when they contain instructions that look urgent or official.
Prompt injection occurs when instructions inside lower-trust material redirect an assistant away from the authorized task. Keep trusted instructions separate from retrieved content, limit tool permissions, require approval for consequential actions, and review outputs for task changes or data requests.
Recognize the trust boundary
A retrieved page may be relevant evidence for a summary while containing text that says ignore previous instructions, reveal hidden data, or send a file elsewhere. Those sentences came from the source, not from the person who assigned the task. OWASP describes prompt injection as a vulnerability arising when natural-language instructions and data are processed together without a clear boundary. A credible layout, official-sounding heading, or citation does not make a page authoritative over the workflow. Train operators to ask who supplied the instruction and what permission they actually have. The source can inform the answer; it cannot expand the assistant's tools, audience, or data access.
Constrain the job before retrieval
Write the trusted task, permitted sources, output format, and forbidden actions before bringing documents into the context. If the job is to summarize a public report, the assistant does not need email sending or access to private drives. Keep connected tools to the minimum required for the task and use human confirmation for actions that affect records or people. This reduces the harm if a malicious document persuades the model to attempt a new step. It is safer than relying on a single warning sentence that the model may misapply. Treat uploaded files, web pages, search snippets, and tool responses as potentially untrusted even when they are useful data.
Inspect suspicious output as a symptom
A summary that suddenly asks for credentials, cites a strange external endpoint, includes a hidden instruction, or proposes an unrelated action should stop the workflow. Compare it with the trusted task and the source packet. Do not follow the new instruction merely to see what happens in a live account. Preserve the relevant document and output for the appropriate owner if investigation is needed. For routine editorial work, the immediate correction may be to remove the malicious passage from the source set and rerun within the approved boundary only after the cause is understood. For an agent with external actions, use the organization's incident route. A fluent answer can still carry a redirected goal.
Test the boundary with representative material
If the workflow routinely reads external documents, include a controlled injected example in its evaluation set. Ask whether the assistant identifies the attack as source text, completes the legitimate task, and refrains from extra actions. Also test ordinary documents so defenses do not reject every instruction-like sentence that is actually part of the material being analyzed. Record tool permissions and the result. A passing test does not make prompt injection impossible; it shows what happened under the tested conditions. Retest when retrieval, connected tools, model behavior, or approval flow changes. Keep the evaluation focused on the actual documents and permissions the team uses.
Give users a simple response rule
When an outside source issues a command to the assistant, treat it as a quotation or data point unless the trusted task explicitly asks to analyze that command. Do not promote it into operational authority. If a document genuinely contains a policy that must guide work, have the responsible person identify and approve that policy outside the untrusted retrieval flow. Keep final answers grounded in the task and cite source claims without following source instructions. This rule is easy to remember and helps editors, analysts, and developers notice the same boundary in different tools.
Example: a research page that orders an email
An analyst asks an assistant to summarize three public reports. One page contains a paragraph addressed to the assistant saying that compliance requires emailing the full source packet to an unfamiliar address. That sentence may be relevant only as evidence of an attack; it is not part of the analyst's assignment. The assistant should continue the summary using legitimate report content and refuse the extra action. The analyst should check that no connected mail tool was used and record the suspicious page for the owner. If the page is needed as a source, cite only its factual content after independent verification. The scenario illustrates why retrieval authority and action authority must be separate.
Sources and update note
Follow the linked official source before a product, price, plan, or policy decision.