Toolformer: A Model That Teaches Itself to Use Tools
How Toolformer builds its own tool-call training data from a handful of API examples and keeps only the calls that improved its predictions.
Contents
Instead of humans writing out every rule for tool use, the model selects the calls that helped and learns from those.
Why language models could not use tools
A large language model (LLM) produces text by predicting the next token. That structure is strong at generating sentences but weak at exact arithmetic and at looking up current facts. One wrong multiplication or one wrong date still comes out inside a perfectly plausible sentence.
One fix is attaching external tools such as a calculator or a search engine. The problem is that a human has to specify when to call a tool and what arguments to pass. Rules multiply as tools multiply, and hand-written rules break easily in new contexts.
The shift Toolformer introduced
Toolformer was proposed in the February 2023 paper 'Language Models Can Teach Themselves to Use Tools', published as arXiv:2302.04761. It makes the model learn on its own when to call an external tool, what arguments to pass, and how to fold the result into the next prediction.
The key point is that no human labels the training data. Given only a handful of application programming interface (API) usage examples, the model generates its own tool-call data in a self-supervised way. Self-supervised means setting a training objective from signals inside the data itself, with no ground-truth labels.
The starting point differs from simply putting tools in the prompt. Toolformer trains the ability to use tools into the model weights. Tool use becomes part of the model's capability rather than an ad hoc technique applied at inference time.
Generating tool-call data without supervision
Toolformer's data generation runs in five stages. Humans supply only the demonstration examples; which calls survive is decided by the model and the loss values.
The diagram traces the data generation flow from demonstration examples through further training. The stages break down as follows.
- Give the model a handful of API usage examples.
- The model produces several candidate calls per position, to be inserted mid-context.
- Each candidate call is actually executed and its result collected.
- Only calls whose results reduce the next-token prediction loss are kept.
- The selected calls are spliced into the text and the model is trained further on it.
Candidate generation splits along two axes. The model first picks several positions in the text where a tool call would fit well. Then, at each position, it produces several calls with different arguments as execution candidates. Since the model proposes both positions and arguments, no human needs to design call rules.
The output of this process is a model that has learned tool use itself. The call positions, the arguments, and the way results feed back are all baked into the training data.
Calls appear inline in the text using a notation. The original paper writes the call and its result together inside square brackets.
Original: 3 times 14 is 42.
Inserted: 3 times 14 is [Calculator(3 * 14) -> 42] 42.The bracketed part is the tool call and its result. After training, the model produces these calls on its own at similar positions. The value 42 in the example is the result of 3 times 14, which anyone can verify.
The usefulness filter
Stage four is the heart of Toolformer. Most candidate calls are useless no matter how many are generated, so filtering down to the ones that helped is what keeps the training data clean.
The filter compares two losses at the same position: the next-token prediction loss with the tool result inserted, and the loss without it. If the tool result reduces the loss by more than a threshold, the call is judged useful and kept.
This criterion lets the model discard unhelpful calls by itself. Calling a calculator in a sentence that needs no arithmetic does not reduce the loss, so it is filtered out. What remains is data consisting only of cases where a tool genuinely improved the prediction.
The selection has a cost. Candidates must be generated at every position, executed, and scored for loss. As the number of tools grows, so does the number of executions, and data generation cost climbs fast.
The tools used in the paper form a narrow set: a calculator, a question answering system, a search engine, a translator, and a calendar. All have clear inputs and outputs, which makes the usefulness of the result easy to measure through loss.
How this differs from ReAct
ReAct is the other prominent pattern for tool use. It is a prompting pattern that alternates reasoning and acting, calling tools at inference time. The two approaches differ from the moment they engage with tools.
| Aspect | Toolformer | ReAct |
|---|---|---|
| Focus | Learns the tool-use ability itself | Combines reasoning and acting at the prompt level |
| Timing | Mainly the training stage | Mainly the inference stage |
| Strength | Can embed a disposition for tool use inside the model | Simple to implement and broadly applicable |
| Limitation | Training and data generation costs are high | Sensitive to prompt quality and loop design |
In practice, the ReAct family is far more widely used, because tools can be attached through prompting alone without retraining the model. Toolformer still matters for turning tool use into a learnable problem.
The boundary with learning environment skills
Toolformer's scope stops at API calls. Learning to act within an environment requires a different approach. The boundary becomes clear next to Self-Driven Grounding, which hypothesizes subgoals, validates them against the environment, and learns reusable skills.
| Aspect | Toolformer | Self-Driven Grounding |
|---|---|---|
| Problem | Learning API and tool use | Learning skills inside an environment |
| Core mechanism | Self-supervised tool-call data generation | Subgoal hypotheses with environment validation |
| Output | Tool-use ability | Reusable skills |
| Application context | Centered on API calls | Centered on environment interaction and action learning |
The two approaches draw their training signal from different places. Toolformer takes it from next-token prediction loss; Self-Driven Grounding takes it from the outcome of interacting with an environment. This is where it becomes apparent that calling a tool and acting in an environment resist a single unified framing.
Limitations and lasting influence
Toolformer reads as a research framework, which constrains using it in production as-is. The main limitations are as follows.
- The API format and the set of tools are relatively fixed.
- Generating and filtering candidate calls is expensive.
- It is less flexible than today's general-purpose tool-calling API ecosystem.
- It resembles a research framework more than production orchestration.
Despite clear limitations, its mark on later work is unmistakable. Treating tool use as part of the model's capability resurfaced in later research on function calling and tool-augmented agents. The idea that a model picks its own tools sits underneath how agents are designed today.
Summary
Toolformer is a framework where the model selects the calls that helped and learns from them, instead of a human writing tool-use rules. At its center is a usefulness filter that generates and executes candidate calls, then keeps only those that reduce next-token prediction loss. The fixed set of tools and the high cost of data generation are clear limits, but turning tool use into a learnable problem is what carried through into later agent research. The ReAct family dominates in practice today, yet reading Toolformer first is the better way to understand where that thinking came from.