Memory poisoning lets stored untrusted content influence later decisions or actions, so writing, retrieval, and execution need separate authorization checks.
Conditions that make memory an attack surface
An agent can store conversations or task records and retrieve them for later requests. The content of a record differs from the authority of its author. Age or a high retrieval score does not make that content trustworthy.
AgentPoison (arXiv:2407.12784) appeared in the NeurIPS 2024 main conference. It evaluates how retrieved examples affect agent behavior when an attacker can insert malicious items into memory or a knowledge base. It does not establish that ordinary queries alone can poison every memory system.
What the experiments support
AgentPoison covers three environments: retrieval-based driving, knowledge question answering, and a healthcare-record agent. It uses embedding space to make triggered queries retrieve malicious examples. The attack avoids retraining the target model, but still requires a path to insert items into the store.
The separate PoisonedRAG study (arXiv:2402.07867) evaluated knowledge-base poisoning at USENIX Security 2025. Together, the studies support the risk that retrieved untrusted material can influence subsequent generation. Their attack success rates cannot be combined as if they used the same experimental conditions.
| Condition to inspect | Meaning | Operational control point |
|---|---|---|
| Insertion path | What can the attacker modify? | Ingestion and write permissions |
| Malicious-item retrieval | Which queries are affected? | Retrieval scope and filters |
| Role in model context | Is evidence interpreted as an instruction? | Context construction |
| Authority to act | Does a wrong response lead to an external change? | Execution gate |
This table translates the findings into application design checks. It does not provide a defensive success rate or a guarantee for all agents.
Separating storage from execution
Saltzer and Schroeder's protection principles describe least privilege and authorization on every access. Applied to memory, these principles mean that stored sentences do not grant execution authority. Policy decisions use author, tenant, and origin metadata recorded by a trusted server.
flowchart LR
I["Untrusted input"] --> W["Check writes and record origin"]
W --> S[("Memory store")]
S --> R["Retrieve within user scope"]
R --> M["Model proposes an action"]
M --> G["Server checks authority, target, and arguments"]
G --> A["Permitted action"]Summarizing origin metadata together with content and raising its trust level would erase this boundary. Keep policy metadata separate from model-generated summaries. Sharing memory across users also requires a separate sharing permission.
Investigation and recovery
For a suspected poisoned item, trace its source, derived summaries, requests that retrieved it, and executed tools. Deleting one record does not establish that copied summaries or caches have disappeared.
| Record | Investigation purpose | Retention consideration |
|---|---|---|
| Record ID and origin | Identify the entry path | Separate it from untrusted content |
| Readers and writers | Check authorization scope | Preserve tenant boundaries |
| Derived-item relationships | Recover copies and summaries | Track deletion scope |
| Tool-execution decisions | Determine actual impact | Minimize stored sensitive arguments |
After recovery, recheck retrieval and actions with the same input. Include legitimate requests alongside attack attempts to detect excessive blocking. This is an operational proposal, not an algorithm that automatically proves causal responsibility.
Summary
AgentPoison and PoisonedRAG show that an attacker-controlled retrieval store can affect later behavior. Their findings do not establish every query-only attack or complete success of a particular defense. Separate memory content, write authority, and execution authority, and enforce checks on the server. Recovery must trace derived summaries and actions already taken as well as the original record.