AgentDojo and InjecAgent: how prompt injection is measured in tool-using agents

The short answer

AgentDojo and InjecAgent are benchmarks that measure how often tool-using agents follow instructions planted in the data they read, the attack known as indirect prompt injection.³ AgentDojo gives agents real tasks in four simulated environments, plants injections in what they retrieve, and checks the environment’s state afterwards with deterministic functions.¹ InjecAgent builds 1,054 cases where a tool’s output carries an attacker’s instruction and counts how often the agent then calls the attacker’s tool.²

Their findings point the same way. In AgentDojo, agents solved under 66% of tasks even without attacks, attacks succeeded against the best agents in under 25% of cases, and restricting the tools available before untrusted data was read cut attack success to about 7.5%.¹ In InjecAgent, ReAct-prompted GPT-4 was vulnerable 24% of the time.²

Two benchmarks, two questions

The key difference is that AgentDojo also measures whether the agent still does its job. A defence that refuses everything would score perfectly on security and zero on utility; AgentDojo shows both.

AgentDojo: tasks checked by state

Each AgentDojo security case pairs a user task with an injection task the attacker wants done instead, and places the injection in data the agent will read while doing the user task. To picture it (our illustration, not a quoted case): the user asks for a summary of unread email, and one email tells the assistant to forward a document to an outside address.

Success isn’t judged by another language model. The authors use deterministic functions that inspect the environment’s state, and they explain why: an LLM judge could itself be hijacked by the injection it’s evaluating.¹ Did the email get sent to the attacker? Look at the outbox. Did the user’s task get done? Check the expected changes.

That gives three numbers:¹

1. Benign utility: task success with no attack.
2. Utility under attack: task success with an injection present and no attacker side effects.
3. Targeted attack success rate: how often the attacker’s goal was met.

InjecAgent: harmful calls counted

InjecAgent is narrower and larger. Each case gives an agent a user instruction that calls a tool; the tool’s response contains an attacker’s instruction; and the test checks whether the agent then calls one of 62 attacker tools. The attacks aim either at direct harm to the user or at stealing private data.²

It also tested a stronger variant: a “hacking prompt” that reinforces the attacker’s instruction. That nearly doubled the attack success rate on ReAct-prompted GPT-4.² The lesson isn’t about one model. It’s that small changes in attacker wording move results a lot, so a single success rate understates what a determined attacker can do.

What the numbers say

The headline results, with the models those papers tested in 2024:

Model numbers age fast; newer models score differently. Two results age slowly.

The tool filter works, with limits. Restricting which tools the agent can call, decided before it reads any untrusted data, was one of the most effective defences. The authors note where it fails: when the tools a task needs can’t be planned in advance, and when the tools the task legitimately needs also enable the attack.¹ An email assistant that must send email can be told to send it to the wrong person.

Detectors help, and aren’t zero. A separate detector cut attack success to 8%. Eight percent of a thousand agent runs a day is eighty.

How to read a prompt-injection score

When a vendor or paper quotes an attack success rate, ask:

1. Against which attacks? A fixed set, or adaptive ones written against the defence? Fixed sets flatter defences.
2. What happened to utility? A defence that halves attacks and halves useful work isn’t a win.
3. How was success judged? State checks are harder to fool than a model’s opinion.¹
4. Which model, which date? Results don’t carry over between models.
5. What was the agent allowed to do? An attack can only succeed through tools the agent has.

The last question is the one under your control in production.

What to take into your own system

These benchmarks measure the model-and-prompt layer. In a deployment, you can also change the layer underneath: what tools exist, and what each call is allowed to do.

• Plan tools before reading data, as the tool filter does. Give the agent the smallest tool set for the task before it opens any email, ticket or page.
• Put rules on arguments, not just tool names. “Send email only to our domain” survives an injection that a tool filter can’t, because the tool is still needed.
• Check state, not transcripts, as AgentDojo does. Know what changed after each run.
• Test with your own tasks. Write cases where the model is assumed fooled and check what its calls could do; our attack suite post describes one approach.

For the history behind the attack, see indirect prompt injection research; for a related benchmark that also scores by database state, τ-bench. The prompt injection entry has the short definition. To see rules on tool calls refuse an action on a real database in your browser, try the playground; to grade what an agent can reach on your machine, run npx deedglass scan (the grade, explained).

Written with AI assistance and checked against our claims register.