Instead of humans writing out every rule for tool use, the model selects the calls that helped and learns from those.
Why language models could not use tools
A large language model (LLM) produces text by predicting the next token. That structure is strong at generating sentences but weak at exact arithmetic and at looking up current facts. One wrong multiplication or one wrong date still comes out inside a perfectly plausible sentence.
One fix is attaching external tools such as a calculator or a search engine. The problem is that a human has to specify when to call a tool and what arguments to pass. Rules multiply as tools multiply, and hand-written rules break easily in new contexts.
The shift Toolformer introduced
Toolformer was proposed in the February 2023 paper 'Language Models Can Teach Themselves to Use Tools', published at NeurIPS 2023 (arXiv:2302.04761). The model learns when to call an external tool and what arguments to pass. It also learns how to use the result in the next prediction.
The key point is that no human labels the training data. Given only a handful of application programming interface (API) usage examples, the model generates its own tool-call data in a self-supervised way. Self-supervised means setting a training objective from signals inside the data itself, with no ground-truth labels.
The starting point differs from simply putting tools in the prompt. Toolformer trains the ability to use tools into the model weights. Tool use becomes part of the model's capability rather than an ad hoc technique applied at inference time.
Generating tool-call data without supervision
Toolformer's data generation runs in five stages. Humans supply only the demonstration examples; which calls survive is decided by the model and the loss values.
flowchart LR
A["API examples<br/>a few demonstration calls"] --> B["Candidate call generation<br/>at positions inside the context"]
B --> C["Tool execution<br/>obtain the real results"]
C --> D["Usefulness filter<br/>compare the loss reduction"]
D --> E["Filtered data<br/>keep only the calls that helped"]
E --> F["Further training<br/>internalize tool use"]The diagram traces the data generation flow from demonstration examples through further training. The stages break down as follows.
- Give the model a handful of API usage examples.
- The model produces several candidate calls per position, to be inserted mid-context.
- Each candidate call is actually executed and its result collected.
- Only calls whose results reduce the next-token prediction loss are kept.
- The selected calls are spliced into the text and the model is trained further on it.
Candidate generation splits along two axes. The model first picks several positions in the text where a tool call would fit well. Then, at each position, it produces several calls with different arguments as execution candidates. Since the model proposes both positions and arguments, no human needs to design call rules.
The output of this process is a model that has learned tool use itself. The training data contains call positions, arguments, and examples of using the results.
Calls appear inline in the text using a notation. The original paper writes the call and its result together inside square brackets.
Original: 3 times 14 is 42.
Inserted: 3 times 14 is [Calculator(3 * 14) -> 42] 42.The bracketed part is the tool call and its result. After training, the model produces these calls on its own at similar positions. The value 42 in the example is the result of 3 times 14, which anyone can verify.
The usefulness filter
Stage four selects useful calls for training. Most candidate calls are useless no matter how many are generated, so filtering down to the ones that helped is what keeps the training data clean.
The filter compares two losses at the same position: the next-token prediction loss with the tool result inserted, and the loss without it. If the tool result reduces the loss by more than a threshold, the call is judged useful and kept.
This criterion lets the model discard unhelpful calls by itself. Calling a calculator in a sentence that needs no arithmetic does not reduce the loss, so it is filtered out. What remains is data consisting only of cases where a tool genuinely improved the prediction.
The selection has a cost. Candidates must be generated at every position, executed, and scored for loss. As the number of tools grows, so does the number of executions, and data generation cost climbs fast.
The tools used in the paper form a narrow set: a calculator, a question answering system, a search engine, a translator, and a calendar. All have clear inputs and outputs, which makes the usefulness of the result easy to measure through loss.
How this differs from ReAct
ReAct is the other prominent pattern for tool use. It is a prompting pattern that alternates reasoning and acting, calling tools at inference time. The two approaches differ from the moment they engage with tools.
| Aspect | Toolformer | ReAct |
|---|---|---|
| Focus | Learns the tool-use ability itself | Combines reasoning and acting at the prompt level |
| Timing | Mainly the training stage | Mainly the inference stage |
| Strength | Can embed a disposition for tool use inside the model | Simple to implement and broadly applicable |
| Limitation | Training and data generation costs are high | Sensitive to prompt quality and loop design |
Prompt-based tool connections in ReAct and training-data construction in Toolformer are different implementation choices. This comparison is not a measurement of their relative adoption.
Limitations and lasting influence
Toolformer reads as a research framework, which constrains using it in production as-is. The main limitations are as follows.
- The API format and the set of tools are relatively fixed.
- Generating and filtering candidate calls is expensive.
- It is less flexible than today's general-purpose tool-calling API ecosystem.
- It resembles a research framework more than production orchestration.
Despite these limitations, Toolformer influenced later work. Treating tool use as part of the model's capability resurfaced in later research on function calling and tool-augmented agents. The idea that a model picks its own tools sits underneath how agents are designed today.
Summary
Toolformer is a framework where the model selects the calls that helped and learns from them, instead of a human writing tool-use rules. At its center is a usefulness filter that generates and executes candidate calls, then keeps only those that reduce next-token prediction loss. The fixed set of tools and the high cost of data generation are clear limits, but turning tool use into a learnable problem is what carried through into later agent research. Evaluate the required model training and tool-execution costs separately before adoption.