Isolating a shared prefix cache starts with defining which requests may reuse each other's cached computation.
A prefix cache reuses attention Keys and Values (KV) when an input starts with a previously processed prefix. Reduced prefill computation can change Time to First Token (TTFT).
The official vLLM security documentation describes timing side channels in shared caches and cache salting. This post explains documented isolation behavior and service checks. It does not assume the performance or security guarantees of a new defense paper.
Response timing and cache sharing
A guessed prefix that matches another user's cached input can change the computation required. Whether an attacker can observe that difference also depends on load, networking, and cache state.
flowchart TD
V[Another user's request] --> C[Shared prefix cache]
A[Candidate prefix request] --> C
C --> H[Cache hit and reuse]
C --> M[Cache miss and computation]
H --> T[Observe time to first token]
M --> T
This diagram explains the observation path. It does not report an attack success rate for a particular environment. Removing sensitive strings from response bodies and limiting timing exposure of shared state require separate evaluation.
Cache keys and trust groups
The vLLM v0.15.0 design documentation describes block hashes using parent hashes, tokens, and extra keys. A request's cache_salt enters the first block hash, restricting sharing to requests with the same salt.
The service must bind authenticated users or tenants to salts. The security documentation recommends sufficiently random secret values instead of predictable account names. Allowing callers to select another tenant's salt defeats the intended boundary.
| Item | What it determines | What it does not replace |
|---|---|---|
| Input tokens and parent hash | Matching prefixes | User authentication |
| cache_salt | Requests permitted to share cached computation | Appropriate group membership |
| Gateway group assignment | Binding an authenticated principal to a sharing group | Security of all model responses and stores |
The third row is a service-design proposal. A vLLM request field does not automatically implement organizational authorization policy.
Isolation scope and cost
Different salts restrict reuse between groups. Further dividing requests that may safely share also reduces reuse opportunities. Measure latency and throughput changes using the actual prefix distribution.
| Configuration | Sharing scope to check | Measurements |
|---|---|---|
| Cache disabled | No prefix reuse | Prefill cost and throughput |
| Global sharing | Requests on the same backend | Cross-group hits and timing distributions |
| Group-specific salts | Requests in one trust group | Reuse within and across groups, and performance |
These configurations propose a local comparison. They provide no universal percentage improvement. Salt isolates cache keys; it does not guarantee removal of every GPU or network side channel.
Service checks
Correct cache partitioning and externally distinguishable timing are separate tests. Similar observed speed does not establish correct isolation.
| Test | Outcome to check |
|---|---|
| Same prefix within one group | Intended reuse occurs |
| Same prefix across groups | Cross-group KV reuse is restricted |
| Missing or forged salts | Authentication-boundary policy is enforced |
| Group changes and secret rotation | New requests use the intended keys |
| Actual load | TTFT distributions, throughput, and cache usage |
These are operational proposals, not benchmarks executed for this post. KV Cache as a First-Class Resource covers capacity and scheduling.
Summary
Shared caches can expose timing differences caused by reuse. vLLM's cache_salt limits reuse to requests with the same value, while the service handles authentication and group assignment. Evaluate isolation and performance costs separately under real request distributions. Cache-key isolation is not a claim that all system side channels have been removed.