RAG Datastore Membership Inference: The Retrieval Corpus as Attack Surface
The decision signals and defensive layers behind RAG membership inference, where a handful of queries reveal whether a document sits in the retrieval corpus.
Contents
Skipping fine-tuning does not help: which documents are in the retrieval corpus leaks through API queries alone.
The target moves from weights to the corpus
Retrieval-Augmented Generation (RAG) keeps sensitive data out of model parameters and stores it separately in an external corpus. That structure produced a common assumption that avoiding fine-tuning also removes the privacy risk. Membership inference research since 2024 shows the conditions under which that assumption fails.
A Membership Inference Attack (MIA) aimed at training data determines whether a particular sample was in the training set, using loss values or token probability distributions. An MIA aimed at RAG asks a different question. Not "was this document used in training" but "is this document in the retrieval database right now". The decision signal is not memorization left in the weights either, but how much the response changes when the retriever attaches that document.
That difference changes the nature of the risk. A training MIA works against a model that has been frozen, so the result is static. A corpus MIA reads the current state, so document additions and deletions are directly observable. On top of that, corpora often hold internal documents, medical records, and legal material, which are more sensitive and more recent than pretraining data.
Anderson, Amit, and Goldsteen (2024) were the first to demonstrate that corpus membership can be inferred from designed prompts alone. It holds in both black-box and gray-box settings (arXiv:2405.20446, as of 2024-05). The fact "this document exists inside the system" leaks out as a side channel.
What the attacker needs
The attacker's observation window splits two ways. Neither requires internal parameters of the model or the retriever.
| Setting | Access | Observed information | Representative work |
|---|---|---|---|
| Black-box | Final natural language response only | response content, entailment relations | S2MIA (2024), MEntA (2026) |
| Gray-box | Response plus per-token output probabilities | probability difference with and without the document | DCMI (2025) |
All that is required is API-level access: send a query to the system and read the response. The moment an internal RAG is opened to an external partner or shipped as a product feature, that condition is already met.
From the defender's side, what is observable is the query text, the retrieval results, and the final response. Which documents the attacker holds as candidates never reaches the server, so the inference attempt itself leaves no trace in the logs.
Failure path: three generations of decision signal
Attack techniques divide into generations by the signal used to decide membership. Each generation needs fewer queries and less preparation than the last.
| Attack | Year | Core signal | Surrogate model | Notes |
|---|---|---|---|---|
| S2MIA | 2024 | similarity and completeness of the response to a designed prompt | not needed | first to demonstrate inference in both black-box and gray-box settings |
| DCMI | 2025 | output probability difference with and without the document (differential calibration) | not needed | outperforms S2MIA and MBA, with the largest gain when retrieval quality is poor |
| MEntA | 2026 | natural language entailment between the response and the candidate document | not needed | up to 0.991 AUC from five queries |
DCMI calibrates on the output probability difference between including and excluding the target document (arXiv:2509.06026, as of 2025-09). It turns a signal that is blurry in response text alone into a sharp one by working with probability distributions.
The same work reports that the improvement over prior attacks is most pronounced when retrieval quality is low. The intuition that a system with weak retrieval is also safer from attack does not hold. That is a concrete reason to keep a thin pilot-stage corpus out of exposed surfaces.
MEntA changes direction and measures whether the model response supports the content of the candidate document through natural language entailment. The reported figure is up to 0.991 AUC from five queries. That is an improvement of up to 0.42 AUC over the previous best and up to a 65x reduction in attack cost (arXiv:2605.24312, as of 2026-05).
Diagram: the decision flow of MEntA-style attacks. The attacker sends a small number of information-seeking queries about the target document. The attacker then converts how strongly the returned response entails the document content into a score.
Operationally the dangerous part is stealth rather than the raw numbers. Earlier generations relied on repetitive, mechanical template queries, which left a pattern visible in the logs. MEntA uses only broad, information-seeking natural language questions. It also needs no surrogate model training, so anyone with API access can attempt it with no preparation stage.
What changes with graph RAG
Liu, Zhang, and Wang (2025) report that graph RAG reduces verbatim leakage of source text but pays for it elsewhere (arXiv:2508.17222, as of 2025-08). The trade-off is that knowledge restructured into entities and relations turns out to be easier to extract.
Compressing and normalizing information into a graph creates a direct path to structural facts of the form "who is related to whom, and how". Privacy analysis conclusions drawn against text-only RAG therefore do not transfer to graph or multimodal architectures. Extending the architecture calls for redoing the threat model.
Defensive structure and its trade-offs
Choi, Park, Byun, Lee, and Park (2025, EMNLP) proposed a defense called MIRABEL (arXiv:2505.22061, as of 2025-05). It detects cases where the similarity between a query and an actual document in the corpus is abnormally high. Thresholds are set with Gumbel-distribution-based anomaly detection. Risky queries are handled in layers, either by limiting what is returned or by adding differential privacy noise.
This defense lowers membership inference accuracy, but not for free. The tighter the threshold, the more legitimate queries from normal users get caught along with the attacks. Defense strength and retrieval recall are two ends of the same dial.
The survey by Xu et al. (2026) organizes RAG privacy defenses into four layers (arXiv:2604.08304, as of 2026-04).
| Defense layer | Method | Limitation |
|---|---|---|
| Retrieval-time blocking | proximity-based query filtering (MIRABEL) | reduced recall on legitimate queries |
| Output perturbation | differential privacy noise (DPVoteRAG, LPRAG) | degraded response quality, privacy budget to manage |
| Encrypted retrieval | distance-preserving encryption (ppRAG), federated learning (FRAG) | compute overhead, deployment complexity |
| Formal guarantees | proof-based architectures (SAG) | few real-world deployments |
The same paper notes that current defenses cluster around trust boundaries after the retrieval step. Governance of the corpus itself and post-hoc detection remain comparatively underdeveloped. Privacy and confidentiality mechanisms are multiplying quickly, but the assessment is that defenses are reactive and fragmented, and therefore weak against adaptive attacks.
Operational checklist
If a RAG system whose corpus contains internal documents or customer data is exposed externally, check the following.
- Rewrite the threat model: discard the premise that "no fine-tuning means it is safe" and register corpus membership itself as an asset to protect.
- Separate corpora: do not mix documents of different sensitivity levels in one index. Splitting indexes per tenant and per level shrinks what a single attack can reach.
- Update detection signals: rules that only look for repetitive, mechanical query patterns miss natural language attacks. Watch the similarity distribution between queries and top retrieved documents as well.
- Layer the defenses: put proximity-based filtering first and add output-side differential privacy for high-risk corpora. Both layers cost recall and quality, so agree on the metrics in advance.
- Observe per account: the query count needed for an attack is small, so an aggregate rate limit is not enough. Also record sudden shifts in the topic distribution of each account's queries.
- Reassess when the architecture changes: moving to graph or multimodal RAG means redoing the risk assessment done for text RAG.
Summary
The privacy boundary of RAG is drawn around the retrieval corpus rather than the model parameters, and that boundary can be probed with API queries alone. The decision signal moved from response similarity through output probability differences to natural language entailment. Neither a surrogate model nor a mechanical template is needed, so defenses that rely on query pattern detection will miss this family of attacks.
The realistic response today is to stack proximity-based query filtering with output-side differential privacy. That choice requires agreeing in advance on how much recall and response quality can be given up. Changing the structure, as graph RAG does, changes the risk profile too, so it is safer not to reuse conclusions drawn from text RAG.