The Tool Layer as the Agent Bottleneck

Once tools grow into the hundreds, the agent bottleneck moves from model reasoning to the tool layer, and selection, trust, authentication, and latency all fail there at the same time.

Discussions about agent performance usually start with which model to use. Measurements from environments with many tools point somewhere else. Failures cluster not in reasoning but in the step that decides which tools become candidates.

This post ties four separate lines of research into one claim. Tool retrieval, tool poisoning, remote server authentication, and parallel execution are treated as separate topics, yet they all act on the same object. That object is the tool record, the single unit holding a name, a description, a schema, and an endpoint. Every figure quoted here was checked against the primary paper as of 2026-09.

Where the bottleneck moved

LiveMCPBench runs 95 real-world tasks over 70 servers and 527 tools (arXiv:2508.01780). It evaluated 12 recent LLMs, and Claude-Sonnet-4 scored highest at 78.95%. Most other models landed in the 30 to 50 percent range.

What matters is not the score but the distribution of failures. The paper reports that retrieval errors account for nearly half of all failures and names retrieval the dominant bottleneck. Swapping in a stronger model changes nothing when the correct tool never entered the candidate list.

This is what changed since Toolformer, which trained tool-use rules into the model itself. When tools arrive dynamically from an external registry, selection stops being an ability inside the model and becomes a retrieval problem outside it. The bottleneck moves outward with it.

The first failure, selection

Three studies converge on the same response. Instead of putting the whole catalog in the prompt, they retrieve and load only the tools a query needs.

Semantic Tool Discovery indexes tool names, descriptions, and parameters as dense embeddings. Across 5 servers, 121 tools, and 140 queries, it cut tool-related token consumption by 99.6% (arXiv:2603.20313). Under the same setup it reached a 97.1% hit rate at K=3, an MRR of 0.91, and sub-100ms retrieval latency.

Dynamic ReAct compared five tool selection architectures in large MCP environments (arXiv:2509.20386). The final choice was a search-and-load structure that narrows candidates first, and it reduced tool loading by up to 50% while holding task completion accuracy.

This structure rests on one premise. The correct tool has to sit inside the top k, and that ranking is decided by the quality of the description text. A weak description keeps a tool out of the candidate list no matter how strong the model is.

Retrieval text as the attack surface

The second problem attaches right here. The description text that decides ranking is a free-form value the server sends, and it enters the model context without validation.

MCPTox targeted 45 live MCP servers and 353 authentic tools. It built 1,312 malicious cases and measured 20 agents (arXiv:2508.14925). The attack success rate for o1-mini was 72.8%, and even Claude-3.7-Sonnet, the model that refused most often, stayed under 3%. The attack drives a legitimate tool toward an unapproved purpose, which is not what content filters are built to catch.

The approval dialog is not trustworthy either. Rashidi's study notes that nothing in the protocol requires the approval view and the bytes handed to the tokenizer to match. The paper calls this the approval-view fidelity gap (arXiv:2607.05744). The Unicode TAG block (U+E0000 to U+E007F) has no assigned glyph in mainstream renderers, so it disappears on screen while the tokenizer reads it verbatim.

Eight concealment techniques were tested over the real MCP protocol with these results.

The last two rows collide directly with the previous section. That no implementation forces re-approval means a tool approved once stays trusted even after its description quietly changes. Retrieval-based dynamic loading, meanwhile, treats a changing tool list as normal operation.

This next point is a judgment. Retrieval ranking and tool poisoning optimize the same field. A description written to rank well is also a description that injects well, and neither goal can be improved alone while both hang on the same text. The attack path itself is covered in MCP Tool Poisoning.

Authentication surface that grows with tool count

There is only one way to reach hundreds of tools. You connect many servers, and a large share of them are remote.

Zhou and colleagues measured 7,973 live remote MCP servers (arXiv:2605.22333). Of those, 40.55% exposed tools with no authentication at all. Authenticated servers are not the comfortable majority.

The authenticated side was no safer. Across 119 OAuth servers that could be tested end to end, every server carried at least one flaw, for 325 flaws in total. Dynamic client registration flaws affected 96.6% of them, and the team obtained 9 CVE IDs.

In remote MCP deployments a server acts as a resource server toward the client and as an OAuth client toward upstream services. When the two roles overlap, who is acting on whose behalf becomes unclear. That is the problem covered in The Confused Deputy. Every added tool adds another instance of this double role.

Approval points erased by latency work

Latency is the fourth problem. More tools mean more calls per task, and in a sequential loop that turns directly into waiting. Recent work attacks the cost along two different axes.

W&D increases width, the number of tools called within a single step (arXiv:2602.07359). Results per scheduler on 100 BrowseComp samples are below.

Descending, which explores broadly early and narrows later, beat the fixed width by 6 percentage points while using fewer turns. Cost falls as well. Calling three tools in parallel per step reached 68% accuracy at 65.7 dollars per 100 tasks, which is 35.9% below the 102.5 dollars the single-tool run required.

PASTE works on the other axis (arXiv:2603.18897). Its full title is Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving. While the model is still generating the next action, PASTE executes the predicted next tool call ahead of time based on recurring patterns. It reduced average task completion time by 43.5% and sped up observed tool execution by 1.8 times.

The prediction accuracy is worth noting. Top-1 accuracy stops at 27.8% while the overall hit rate reaches 93.8%. The structure works because discarding a wrong prediction costs less than the gain from a correct one.

That axis therefore runs straight into the approval problem. Speculative execution calls a tool before the model confirms it, so it needs a separate filter for actions with side effects. PASTE detected 602 potentially side-effecting speculative actions among over 20,000 and blocked them before commit.

Where the four problems overlap

All four problems make different demands on a single tool record.

• The four paths read one source, so editing a description shifts retrieval rank and attack surface at the same time.
• When indexing, approval, token issuance, and execution happen at different moments, each path holds a different version of the same record.
• The 0/8 result on forced re-approval means the protocol ships no default that catches this version mismatch.

The object each problem manipulates and the point where it collides are summarized below.

Counterexamples and limits

There are conditions under which the claim does not hold. With fewer than ten tools, all first-party, and a fixed list, no retrieval layer is needed and the bottleneck returns to the model. A network-isolated internal deployment is similar. Much of the authentication risk is bypassed by removing exposure itself, and the authentication axis loses its force there.

What the sources do not answer is equally clear. No study measuring all four problems in one system was found. The way this post combines them is a judgment drawn from placing the results side by side, and the size of the combined effect has not been measured.

Individual figures also have narrow scope. The LiveMCPBench retrieval failure share comes from that benchmark's harness configuration, and the PASTE numbers come from one serving stack and three workload families. The authentication measurement draws its population from public remote servers discoverable through search engines. It does not represent authentication levels in internal deployments.

Deciding where to start

Scale decides which axis to solve first.

Regardless of order, the following items are worth checking.

• Verify that the bytes shown in the approval view match the bytes handed to the model.
• Bind a digest of the tool definition to the approval record and require re-approval when the value changes.
• Run static validation alongside every index refresh so the index and the approval state do not drift apart.
• Separate tokens per server and validate audience so one token does not flow to several servers.
• Enable speculative execution and wider parallel calls only for read tools that carry no side effects.

Summary

In an agent with hundreds of tools, the tool layer rather than model reasoning decides the outcome. LiveMCPBench reported that nearly half of failures occur during retrieval, and the description text that retrieval ranks is also the channel poisoning travels through. Reaching that count requires remote servers, 40.55% of which had no authentication, and the width and speculation that cut latency outrun human intervention. The four problems pull the same tool record in different directions, so the retriever, the approval view, the credential store, and the scheduler belong in one layer. For a small, fixed tool list the claim does not apply, and model choice remains the largest variable there.