jongkwan.dev
Development · Essay №102

LLM Agent Memory Poisoning

A delayed attack that corrupts an agent's long-term memory using nothing but ordinary user queries, and the defenses that are shifting toward post-hoc auditing and origin binding.

Jongkwan Lee2026년 7월 10일9 min read
Contents

Memory poisoning corrupts an agent's persistent store, separating the moment of attack from the moment the damage surfaces.

Why long-term memory becomes an attack surface

An LLM agent keeps past conversations and task records in a persistent store such as a vector database. When a new request arrives, similarity search pulls some of those records back into context. That store is what the literature calls long-term memory. Retrieved records land in the same context window as the sentence the user just typed, and nothing in that window tells the model which of the two deserves more trust.

MCP tool poisoning, which corrupts tool description text, misfires at the moment of the call. Memory poisoning differs in that a session boundary sits between corruption and effect. An attacker plants a record today, and the damage can appear weeks later in a different user's session.

Lin et al.'s survey of long-term memory security (arXiv:2604.16548, as of 2026-04) organizes memory threats along three axes: persistence, statefulness, and propagation. Persistence is the property that a planted record survives across sessions. Statefulness is the property that agent behavior depends on accumulated memory state. Propagation refers to corrupted memory spreading to other agents through a shared store in a multi-agent setup.

The hardest condition to work with is privilege. An attacker needs no system access and no retraining opportunity; simply exchanging queries as an ordinary user is enough to leave records in the store.

The attack lineage: from AgentPoison to MINJA

Comparing three key papers by threat model and reported numbers shows how far the barrier to entry has dropped. All figures below are values each paper reported from its own experimental setup.

StudyVenueCore techniqueThreat modelReported success rate
AgentPoison (arXiv:2407.12784, as of 2024-07)NeurIPS 2024Inject malicious documents into memory or the knowledge base, designed so trigger queries map into a specific embedding regionRequires document insertion, no retrainingASR above 80% at a poison ratio under 0.1%, with under 1% effect on benign queries
MINJA (arXiv:2503.03704, as of 2025-03)NeurIPS 2025Induce a bridging step, then progressively remove the inducing textQueries and observations only (query-only)98.2% injection success rate, 76.8% ASR
Replication study (arXiv:2601.05504, as of 2026-01)2026-01Reproduce MINJA on GPT-4o-mini, Gemini-2.0-Flash, and Llama-3.1-8Bquery-onlyInjection success rate above 95% under ideal conditions, 70% ASR

AgentPoison assumed the attacker could place documents into the store. MINJA removes that assumption and reaches the same goal through the normal conversational interface alone.

MINJA works in two stages. First the attacker uses an inducing prompt to get the agent to generate its own intermediate reasoning step. That step, called a bridging step, links the victim query to the malicious conclusion. Then the attacker shortens that inducing prompt bit by bit (progressive shortening) until the record is retrieved naturally without any inducing text at all.

Operationally the second stage is the worse one. The final record left in the store contains almost no explicit attack wording, so post-incident investigation that greps logs for keywords or patterns turns up nothing.

There is one mitigating factor. The same replication study reports that the attack loses much of its effect under realistic conditions where legitimate memory has already accumulated. The vulnerable window is right after a new deployment, or any period where per-user memory is sparse.

Failure points along the memory lifecycle

The survey cited above divides agent memory into a six-stage lifecycle. Splitting it into stages makes clear that the point where corruption enters and the point where damage appears are not the same.

Diagram description: a flowchart grouping the six-stage memory lifecycle into three segments. Corruption enters through Write and Store, damage surfaces through Retrieve and Execute, and spread and recovery are handled by Share and Propagate and by Forget and Rollback.

Each segment offers different controls and leaves different risk behind.

  • Write and Store decide what gets remembered. Any record not filtered out here becomes a candidate for every session that follows.
  • Retrieve is the bottleneck where a poisoned record turns into actual damage. A planted record that is never retrieved is harmless.
  • Execute is where retrieved text becomes action, which makes it the last place to gate destructive operations such as outbound transmission or file manipulation.
  • Share and Propagate is where single-agent defense stops working. Corruption in one agent transfers to another through shared memory.
  • Forget and Rollback is the recovery path after an incident. Without a decided policy on which records get deleted and when, incident response stalls entirely.

The survey groups defensive goals into four axes: integrity, confidentiality, availability, and governance. Memory poisoning targets integrity directly and spreads into a governance problem through the propagation path.

Three generations of defense

Defensive research splits three ways according to when it intervenes. Real-time filtering, post-hoc causal auditing, and origin binding appeared in that order, and they are not substitutes for one another.

Real-time filtering. Sunil et al. (arXiv:2601.05504, as of 2026-01) propose mediating memory reads and writes with a trust score built from multiple signals. The store is kept clean with temporal decay and pattern-based filtering. The same paper reports the limitation: raising the trust threshold blocks legitimate memory, and lowering it lets subtle attacks through.

Post-hoc causal auditing. MemAudit (arXiv:2605.23723, as of 2026-05) chooses tracing over blocking. Once harmful behavior has been observed, it looks for which stored memory caused it. It uses two mechanisms.

  • Counterfactual memory influence score: measures whether the harmful output disappears when a given item is removed, yielding a causal contribution estimate.
  • Memory consistency graph: treats the whole store as a graph and identifies structurally anomalous items that are inconsistent with the rest.

MemAudit reports lowering the ASR of QA attacks from 70% to 0%, and of reasoning agent (RAP) attacks from 83.3% to 0%, in its evaluation. It also reports defensive effect against MINJA. That is exactly where a causal approach meets the MINJA family, which leaves no explicit attack wording behind.

Origin binding. Louck (arXiv:2606.24322, as of 2026-06) attempts structural containment instead of detection. Non-malleable, origin-bound authority is a design that binds a memory item inseparably to the origin that produced it. It eliminates the path for tampering with an item or reusing it in a different context. For validation, the authors report building a benchmark called MEM-INV-Bench and confirming defensive effect against existing poisoning attacks.

The three generations compare as follows on intervention point and limitation.

GenerationIntervention pointWhat it blocksLimitation
Real-time filteringWrite and RetrieveStorage and retrieval of low-trust recordsOver-blocking and misses, depending on threshold tuning
Post-hoc causal auditingAfter ExecuteIdentifying and removing the record that caused the damageCan only intervene after damage has occurred once
Origin bindingWrite and Store designItem tampering and out-of-context reuseThe specifics of the formal verification are not confirmable from public material

The Louck paper claims machine-checked guarantees. The published abstract and body summary alone do not confirm whether the work used Coq, Isabelle, or another formal verification tool. Anyone evaluating adoption should verify this directly.

Defense for multi-agent propagation scenarios is still at the stage of survey-level framework proposals. No defense validated in an actual multi-agent deployment has accumulated in any of the three branches above.

Operational checklist

Even where the published defense techniques are not directly deployable, several controls can be set as policy at the memory layer.

  • Require origin, authoring session, and authoring principal as mandatory metadata on every memory record. Neither post-hoc auditing nor origin binding can start without this information.
  • Block memory retrieval across user and tenant boundaries by default, so a record planted by one user's query never becomes a candidate in another user's session.
  • Give memory a retention period and a decay rule. Indefinite retention hands the attacker the persistence condition for free.
  • Mark retrieved memory distinctly from user input in the context, and treat memory-derived text as reference material rather than instructions.
  • Never execute destructive actions such as outbound transmission, payment, or file deletion on memory-derived grounds alone; require human approval.
  • Build the recovery procedure in advance: delete a specific named record, then re-verify subsequent behavior.
  • Apply conservative retrieval thresholds during the post-deployment window when per-user memory is still sparse.

All of these can be enforced in the memory layer and orchestration code without changing the model.

Summary

Memory poisoning corrupts an agent's persistent store using queries alone, with no system access, and causes damage in arbitrary later sessions. MINJA progressively removes its inducing prompt so no explicit attack wording remains in the store, and reports a 98.2% injection success rate with 76.8% ASR.

Defense is moving from real-time filtering toward post-hoc causal auditing and origin-binding structures. Each carries its own limitation: threshold tuning, intervening only after the fact, and unverified formal specifics. The controls available today are origin metadata on memory records, isolation at user boundaries, retention limits, and human approval for destructive actions. Defense against memory propagation between agents remains under-validated, so it is better to define isolation boundaries first when introducing shared memory.