The Agent Memory Trilemma

In agent long-term memory, the measures that save tokens and the measures that protect accuracy and security pull the same lever in opposite directions.

The central decision in agent memory design is not what to remember but what to compact. Keeping records verbatim inflates the tokens read on every query. Summarizing them to save cost lowers the accuracy ceiling and the ability to trace what happened afterward. Overlay the 2026 memory architecture research with the memory security research, and these three axes turn out to be bound to a single point rather than varying independently.

Three Axes Tied to One Decision

Token cost, accuracy, and attack surface are usually treated separately. Cost belongs to infrastructure, accuracy to retrieval quality, and attack surface to security, so different teams watch each one. Yet a single manipulated variable moves all three. That variable is how much of the original a system discards at write time and replaces with a summary.

The right column is not uniformly better. Reducing compaction buys accuracy but takes on cost and retention risk at the same time. Whichever way the lever moves, only two of the three axes are satisfied, which is what makes this a trilemma.

Two Token-Saving Methods with Different Costs

The Mem0 paper (arXiv:2504.19413, verified 2026-09) measures full-conversation input against memory-based retrieval on the LoCoMo benchmark. The paper reports the following figures.

Cutting tokens by 93 percent cost 6.02 points of judged score. The paper itself notes that full-context input holds a slight accuracy edge, and presents the 92 percent latency reduction as a practical trade-off. What matters here is not the savings rate but what was sold to obtain it.

Another design reaches similar savings by a different route. MemGPT (arXiv:2310.08560, verified 2026-09) does not replace originals with summaries. It pages archive content into a fixed-size core context through function calls, one bounded page at a time. The token cost of a single retrieval becomes independent of total archive size, and on the DMR task GPT-4 accuracy rose from 32.1 percent to 92.5 percent.

The two approaches look comparable in savings but carry different prices on the other two axes. Pagination keeps the original and limits only how much is read at once, while summarization discards the original. This distinction determines everything that follows in this article. Comparing savings rates without comparing original retention creates the illusion that both approaches cost the same.

When Summarization Cuts Accuracy

Summarization does not always reduce accuracy. RAPTOR (arXiv:2401.18059, verified 2026-09) recursively clusters and summarizes chunks to build a bottom-up tree. With this structure, paired with GPT-4, it reached 82.6 percent on the QuALITY benchmark against a previous best of 62.3 percent. Hierarchical summarization raised accuracy by more than 20 points in that case.

Leaf retention is what makes the difference. RAPTOR retrieval does not traverse the tree from the top. It flattens every layer into one pool and selects top-k from it. Higher summary nodes sit on top as an index into the original chunks rather than replacing them. Because the originals remain in the store, summarization shortens the retrieval path without shrinking the evidence available for judgment.

The paper's hallucination analysis shows the same structure. Hand annotation of 150 nodes found minor hallucinations in 4 percent of them, six nodes. Those did not propagate to parent nodes and had no discernible effect on question answering. The common belief that summarization errors spread upward is not supported by this paper.

The 6 points Mem0 gave up, by contrast, are the price of discarding originals. Once only atomic facts remain, there is no sentence to return to and verify. Summarization damages accuracy not because a summary is produced but because the summary replaces the source.

Where a Compaction Threshold Becomes an Attack Parameter

The security axis sharpens this distinction considerably. MPBench, from Dash and colleagues (arXiv:2606.04329, verified 2026-09), builds 3,240 test cases across six attack classes and seven domain types. It also measures the false positive rate against 2,997 benign examples. The abstract states the conclusion in one line. Agents designed to write and retrieve memory more aggressively are more exploitable.

Measurements on two real agents support that sentence. Average attack success rate was 34.25 percent on OpenClaw and 66.67 percent on HERMES. The paper attributes the gap to the write design of HERMES. Its retention policy writes more often, and its memory compaction threshold sits low at 2,200 characters.

The highest success rate on HERMES came from Salience-Driven Compaction, which manipulates the salience judgment inside the compaction step. The paper explains that a more permissive write policy and a lower compaction threshold together create this condition. A threshold lowered to save tokens became a parameter available to the attacker.

Existing prompt injection detectors also struggle against this family. Among the four detectors measured, PromptArmor caught 84.44 percent of strong-signal attacks but fell to 42.50 percent on weak-signal ones. That is a gap of 41.94 percentage points, and it narrowed only after retraining for memory poisoning. LLM Agent Memory Poisoning covers the attack lineage and the generations of defense, so this article does not repeat them.

Provenance as the Link Between the Axes

Placing the evidence so far side by side reveals a common term. The accuracy axis needs a path back from the summary to the original, and the security axis needs a value that fixes where a record came from. Both are provenance.

The diagram places the compaction step of the write path at the center and puts the gains and losses from that same step next to each other. Compaction lowers tokens while discarding the source identifier and the author, and it becomes a new target in its own right.

The long-term memory security survey by Lin and colleagues (arXiv:2604.16548, verified 2026-09) addresses this point directly. Its conclusion is that long-term memory security cannot be retrofitted at retrieval or execution time. It must be anchored from the outset in storage-time provenance, versioning, and policy-aware retention. Provenance is not a field that can be added after an incident.

The TMA-NM work by Louck (arXiv:2606.24322, verified 2026-09) formalizes why. It identifies three channels that launder an untrusted origin, and the first of them is the agent's own summarization. After a summary, the content looks benign and the derivation edge flips to trusted. Across a benchmark of eight frontier models, existing defenses permitted up to 68 percent laundering attack success. TMA-NM, which binds origin at write time, recorded 0 percent across all models and channels.

Summarization is the standard instrument of token savings. From a security view it is a laundering channel, from an accuracy view a point of evidence loss, and it is an attack surface in itself. That is why the three axes meet at compaction.

Conditions Where This Frame Does Not Hold

To keep the claim from widening, here are the points the sources do not support.

First, volume of records and attack surface do not increase together monotonically. The study reproducing MINJA (arXiv:2601.05504, verified 2026-09) confirmed injection success above 95 percent and attack success of 70 percent under idealized conditions. Under realistic conditions with legitimate memories already accumulated, however, it reports that attack effectiveness drops sharply. When legitimate records are plentiful, a poisoned record is diluted in top-k retrieval. What raises risk is the aggressiveness of the write policy, not stored volume by itself.

Second, propagation of summarization errors is an unverified assumption. As shown above, RAPTOR hallucinations did not spread to higher layers and did not affect question answering. This paper gives no basis for saying that hierarchical summarization automatically accumulates errors.

Third, the relationship between compaction threshold and attack success is not a controlled experiment. MPBench compared two agents, and the two differ in retention policy and system prompt as well as in threshold. The paper reports no ablation that varies the threshold alone.

Fourth, the 6-point gap Mem0 accepted holds only under specific conditions. LoCoMo conversations run around 26,000 tokens, and once a conversation exceeds the context window, full-conversation input disappears as an option. At that point the baseline for accuracy comparison changes.

Compaction Exemption Criteria

The design decision is not whether to compact but which layer to leave out of compaction. Layers differ in lifetime and purpose, so there is no reason to apply one policy to all of them.

Compacting working context is comparatively cheap. Intermediate reasoning already applied, or raw grep output, can be dropped because the original remains in the store. Compacting semantic and procedural memory is the expensive case. Discarding originals there loses accuracy and audit evidence at once.

Recording failure analyses and consulting them on the next attempt, as in Reflexion, is the typical shape of episodic memory. If only the distilled lesson survives, there is no way to check which execution produced it. Recording the source trace identifier alongside it preserves the evidence while keeping the same summary.

Shared memory across multiple agents adds one more criterion. A shared store is the path by which one agent's contamination reaches another. Restrict write permission and read scope per agent, and express conflicts as multiple claims with validity periods instead of overwriting.

Failing to separate absent evidence from contradicted evidence makes the verification itself return wrong answers. Where to draw this boundary connects to the state management problem covered in LangGraph Agent Orchestration.

Three checks come first in practice, and this is a judgment drawn from the studies above. Check whether the compaction step preserves a source identifier, and whether the compaction threshold is a parameter exposed to attackers. Then check whether summary records stay distinguishable from original records in retrieval results.

Summary

Token cost, accuracy, and attack surface in agent long-term memory all hang on compaction, and the single value that collapses all three is provenance. In the Mem0 LoCoMo results, a 93 percent token reduction cost 6.02 points of judged score. In MPBench, the agent with the lower compaction threshold was breached most heavily by compaction-stage poisoning at 85.17 percent. Keeping the original while reducing only how much is read, as RAPTOR and MemGPT do, captures much of the savings without the accuracy loss. The question to ask at design time is not how much to reduce but whether a path back to the original survives the reduction.