AgentPoison: how a few poisoned memory entries can steer an AI agent

The short answer

AgentPoison is a 2024 attack that targets what an agent remembers rather than what it’s told. The authors poisoned agents’ long-term memory or retrieval knowledge base with a few crafted entries keyed to an optimised trigger. When a later input contained the trigger, the agent retrieved the malicious entries and followed them; without the trigger it behaved normally.¹

Against three agents, a self-driving planning agent, a question-answering agent and EHRAgent, a healthcare records agent, they report an average attack success rate above 80% for each, with under 1% impact on normal performance and under 0.1% of the memory poisoned.¹ OWASP’s agentic list calls this class of risk ASI06, Memory and Context Poisoning.² Defences start with controlling who can write to an agent’s memory, and continue with checks on what the agent then does.

How agent memory works

Many agents improve by remembering. Two common designs:

• Long-term memory of past cases. The agent stores tasks it solved, with the steps it took, and retrieves similar ones as examples for new tasks.
• A retrieval knowledge base (RAG). The agent searches documents or records and adds the matches to its context.

Both work by similarity search: the input is turned into a vector, and the nearest stored items come back as context. Whatever comes back, the model treats as relevant guidance. That’s the trust AgentPoison exploits.

Memory is attractive to builders for good reasons. It lets an agent reuse solutions that worked, follow an organisation’s conventions, and improve without retraining. But it also means the agent’s behaviour depends on a store that changes over time, often written by the agent itself, by other agents, or by imports from documents and tickets. Every one of those writers is a possible way in, and most teams don’t review memory the way they review code.

How the attack works

The attack has two parts.¹

1. Optimise a trigger. Using constrained optimisation, the attacker finds a trigger that, when present in an input, makes retrieval return the attacker’s entries rather than normal ones.
2. Plant a few entries. The attacker adds a small number of memory or knowledge-base entries that pair the trigger with the behaviour they want, such as a dangerous action.

No model training or fine-tuning is needed.¹ The attacker only needs to get a handful of records into the store, and to get the trigger into a later input, for example through a document, a ticket or a message the agent reads.

That second step links AgentPoison to indirect prompt injection, where instructions arrive through data the application retrieves.³ AgentPoison moves the payload one step earlier, into the store the agent consults.

What they found

Across them, the authors report under 1% impact on benign performance and a poison rate under 0.1%.¹ In other words, the store can be overwhelmingly clean, normal behaviour can look unchanged, and the attack still works when the trigger appears.

The EHRAgent row is the reason this belongs in a healthcare discussion. An agent that answers questions over patient records by writing code or queries, guided by remembered examples, can be steered by examples someone planted. The concern here is data integrity and system safety, not clinical judgement.

Why it’s hard to notice

Three properties make memory poisoning quieter than a direct attack:

1. Normal tests pass. With benign performance barely affected, evaluation on ordinary inputs shows nothing.¹
2. The payload is tiny. Under 0.1% of entries is easy to miss in a review of a large store.¹
3. It persists. Unlike a prompt injection in one conversation, a poisoned memory entry waits for every future input that carries the trigger.

OWASP describes ASI06 the same way: adversaries corrupt or seed stored context so that future reasoning, planning or tool use becomes unsafe.²

Defences by layer

The action layer is worth stressing because it doesn’t depend on detecting the poison. If the planted entry tells the agent to delete records, a rule that refuses bulk deletes, or an approval step for them, holds regardless of where the instruction came from. That’s the role a tool-call policy plays. It’s also the layer our own product works at, and the limit applies: a gate on actions doesn’t see memory, so it can’t stop an agent from giving a wrong answer based on a poisoned entry. Our OWASP mapping marks ASI06 as not covered for that reason.

A checklist for agents with memory

1. List every store the agent reads from: memory, RAG indexes, example banks.
2. Restrict writes to trusted processes, and log who wrote each entry.
3. Keep provenance on entries and show it when they’re retrieved.
4. Review new action-describing entries before they become retrievable.
5. Put rules and approvals on high-impact actions, so a poisoned instruction can’t complete a dangerous step alone.
6. Test with planted entries in a copy of your store, the way the paper does.

For how injected text reaches agents in general, see indirect prompt injection research; for agent designs that keep untrusted data away from decisions, CaMeL and the six patterns. To see rules refuse an action on a real database in your browser, try the playground; to grade what an agent can reach on your machine, run npx deedglass scan (the grade, explained).

Written with AI assistance and checked against our claims register.