Prompt injection defenses vs adaptive attacks: what a 12-defense study found

The short answer

Mostly not, when the attacker adapts. In October 2025, Milad Nasr, Nicholas Carlini and twelve co-authors tested 12 recent defenses against jailbreaks and prompt injection, using attacks tuned to each defense’s design: gradient methods, reinforcement learning, random search and human-guided exploration. Against most of the 12, attack success was above 90%, even though most of those defenses had originally reported near-zero attack success.¹

Their recommendation is about evaluation: future defense work must consider stronger attacks.¹ The lesson for anyone deploying agents is about design: don’t rely on a model-level defense to keep a fooled agent from acting. Limit what its tool calls can do.

Why defenses look better than they are

The paper’s diagnosis is that defenses are often evaluated against a fixed set of attack strings, or against general optimisation attacks that weren’t designed to beat that particular defense.¹ A fixed test set measures how well a defense handles yesterday’s attacks. An attacker in the real world sees the defense and adjusts.

Security people know this pattern from other fields: a lock is tested by someone trying to pick it, not by checking it against a list of keys. The title says it in four words: the attacker moves second.

What was tested

The twelve defenses span the main approaches in the literature:¹

The headline result, above 90% attack success against most of the 12, cuts across these families. No approach was safe just because of the kind of defense it is.

How the attacks adapted

Four methods, scaled up and tuned per defense:¹

1. Gradient descent, where model weights are available, to find inputs that push the model toward the attacker’s goal.
2. Reinforcement learning, training an attacker model to write injections that work.
3. Random search, mutating candidate injections and keeping what scores better.
4. Human-guided exploration, people iterating with feedback.

The common thread is feedback. Each method observes whether an attempt worked and uses that to improve the next. A defense evaluated without that loop hasn’t met the attacker it will face.

People beat every challenge

The study also ran a human red-teaming competition with over 500 participants, 40 challenges spanning several defenses, models and AgentDojo tasks, and a prize pool of 20,000 US dollars. Competitors defeated every challenge.¹

AgentDojo matters here because it’s an agent benchmark with tools, where success is judged by the state of the environment, not by text.² So these weren’t only chatbot jailbreaks; some were agents doing things they shouldn’t, in a tool-using setting.

Questions to ask about any defense

Whether it’s a paper, an open-source filter or a vendor’s product, these questions separate a measured claim from a hopeful one:

1. Who attacked it? The defense’s authors, or someone trying to break it?
2. Did the attacks adapt? Were they tuned against this defense, with feedback, or drawn from a fixed list? The study’s whole point is that the second kind overstates robustness.¹
3. How much effort was spent? Ten minutes of prompting and a week of optimisation give very different numbers.
4. What happens when it fails? If the defense is bypassed, what can the agent then do? That answer comes from the system design, not the defense.
5. Is the number from today’s model? A defense measured on one model may behave differently on the next.

A defense that has good answers to the first three is worth having. The fourth question decides whether you can survive the day it fails anyway.

What it doesn’t say

Reading this paper as “nothing works” would be a mistake. It’s about a specific kind of defense: ones that try to make the model or a filter recognise and resist injected text. It doesn’t evaluate system designs that keep untrusted text away from decisions altogether, such as CaMeL, which separates a planning model from a model that reads untrusted data and checks a policy before every tool call.³ CaMeL’s own authors are careful too: they say prompt injection attacks are not fully solved.³

It also doesn’t mean model-level defenses are useless. They raise the cost of casual attacks and cut the rate of accidental instruction-following. They just can’t be the last line.

What to do if you build agents

Assume a determined attacker will eventually get text past any filter and into your agent’s decisions. Then make sure that moment isn’t catastrophic.

1. Limit tools per task. An agent that can’t send email can’t be talked into sending it.
2. Put rules on arguments, enforced outside the model. “Only our domain”, “only these tables”, “no more than 20 rows”. A rule doesn’t care how persuasive the injection was.
3. Separate reading from acting where you can, as the design-pattern research suggests. Our post on CaMeL and the six patterns compares the options.
4. Ask a person before consequential writes, and keep those prompts rare enough to be read.
5. Test adaptively yourself. Don’t stop at a list of known attack strings. Give someone the defense and a goal, and see how long it takes. For your own tool layer, write cases that start from “the model is already fooled”, as in our attack suite post.
6. Record every tool call, so a successful attack is visible afterwards.

The prompt injection entry gives the short definition; AgentDojo and InjecAgent explain how attack success is measured, and the OWASP agentic list, mapped shows where a tool-call gate fits among the other controls. To see a rule refuse a call on a real database in your browser, try the playground; to grade what an agent can reach on your machine, run npx deedglass scan (the grade, explained).

Written with AI assistance and checked against our claims register.