Prompt Injection
Hiding instructions inside content a model reads, so it follows the attacker instead of you — and the reason is structural, which is why it isn't fixed.
When not to use it
- (This is a risk, not a technique — the equivalent question is when you can stop worrying about it.)*
- When nothing the model reads comes from outside your trust boundary. Rare, and check the assumption twice.
- When the model has no capabilities. A pure text generator with no tools and no side effects can be injected to no consequence beyond a bad answer.
- Never, if it has tools and reads external content. There's no configuration that makes this safe.
Reach for something else instead
- (Ways to make it not matter, since you can't prevent it.)*
- Capability restriction — the only reliable defence. Don't grant what you can't afford to lose.
- Human confirmation on irreversible actions — moves the trust boundary to a person.
- Read-only credentials — worst case becomes a wrong answer rather than a wrong action.
- Treating model output as untrusted — validate it before it reaches anything that acts.
Read more on the blog
- What is prompt injection, and why is it unsolved?Prompt injection is the number one security risk for AI applications, and researchers treat it as unsolved: not a bug waiting for a patch, but an architectural flaw in how language models work. Here is why a model cannot reliably tell instructions from data, why that makes AI agents dangerous, and what actually reduces the risk.
- How to secure an LLM application: risks and defensesTraditional security rests on a boundary between instructions and data. A language model has no such boundary, because both arrive as one stream of text it cannot tell apart. That single fact determines every defense that works, and every one that does not.
- Why AI agents fail: the seven failure modesGartner predicts over 40% of agentic AI projects will be canceled by 2027. The failures follow patterns, seven of them. The taxonomy: what breaks, why, which real incident proved it, and which control would have prevented it.
- Agent permissions: the question nobody asks until afterwardsEighty percent of organisations running agents say those agents have taken unintended actions. One in five has had a security incident from one. Both wrong answers to "whose credentials" are still the common ones.
Further reading
- Greshake et al. (2023), Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — the systematic treatment; read this one.
- Wallace et al. (2024), The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions — the most credible mitigation direction, and honest about being a tendency rather than a boundary.
- Perez & Ribeiro (2022), Ignore Previous Prompt: Attack Techniques For Language Models — early formalisation of the attack.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Believing a system prompt is a security boundary. It's a strong suggestion.
- Defending against direct injection and ignoring indirect, which is the one that hurts your users.
- Testing with obvious attacks ("ignore previous instructions") and concluding you're safe. Real attacks don't announce themselves.
- Adding a detection classifier and calling it solved. The attacker iterates; your classifier doesn't.
- Assuming multi-agent architectures contain the blast radius. They enlarge it — every hop is another chance for data to become instruction.
At a glance
Often compared with
Where this sits
35 concepts come first. Understanding it opens up 1 more.
Computed from the prerequisite graph, not assigned. How this works