ReAct (Reasoning + Acting)

An agent pattern that cycles through Thought, Action, and Observation, reinforcing reasoning with external tools.

Concept

ReAct is an agent framework that combines reasoning and acting. It follows a loop: think, call an external tool, then observe the result.

flowchart LR
    T["Thought<br/>plan next action"] --> A["Action<br/>call tool"]
    A --> O["Observation<br/>receive result"]
    O --> D{"Goal reached"}
    D -->|no| T
    D -->|yes| F["Final answer"]

How the loop works

1. Thought: analyze the current situation and plan the next step
2. Action: call an external tool (search, calculation, and so on)
3. Observation: receive the result of the action
4. Repeat steps 1-3 until the goal is reached

Example

The search responses below are hypothetical illustrations, not current information or actual tool-execution results.

Question: "How tall is Lotte World Tower?"

Thought: I need the height of Lotte World Tower. A search is required. Action: [Search] "Lotte World Tower height" Observation: 554.5 meters

Thought: I have the information. I can answer. Answer: 554.5 meters

Multi-step reasoning example

Question: "What is the height difference between the tallest and second-tallest buildings in Korea?"

Thought: First I need to know the tallest building. Action: [Search] "tallest building in Korea" Observation: Lotte World Tower (554.5m)

Thought: I also need the second-tallest building. Action: [Search] "second tallest building in Korea" Observation: Haeundae LCT The Sharp (411.6m)

Thought: Now I can compute the difference. Action: [Calculate] 554.5 - 411.6 Observation: 142.9

Answer: 142.9 meters

Advantages

Problems Chain of Thought (CoT) alone cannot solve

Question: "What is the weather in Seoul today?"

CoT only: Thought: Seoul is the capital of Korea, and... Answer: Seoul has four distinct seasons... (hallucination)

With ReAct: Thought: I need real-time weather information Action: [Search] "Seoul weather today" Observation: Clear, temperature -2°C Answer: Seoul is clear today, with a temperature of -2 degrees Celsius.

Performance

ReAct is not always best on its own. On HotpotQA, which is reasoning-heavy, CoT does better, and ReAct performs best when combined with CoT Self-Consistency (CoT-SC). On FEVER, a fact-verification benchmark, ReAct beats CoT. The numbers below are the PaLM-540B results from Table 1 of the ICLR 2023 paper.

The arrows indicate switching order under the conditions specified in the paper. These are different policies. The results cannot establish a ranking for all reasoning or fact-verification tasks.

Available tools

Limitations

1. Failure on complex multi-step tool use

   • Errors increase as the tool call sequence gets more complicated

2. Hard to recover from a wrong path

   • Once headed in the wrong direction, backing out is difficult

3. Tool dependence

   • Without an appropriate tool, the problem cannot be solved

Combining with Reflexion

ReAct's limitations can be mitigated by combining it with Reflexion:

First attempt Thought: Let me search Action: [Search] "a poor query" Observation: Irrelevant results (failure)

Reflexion: "The query was too vague. I should search more specifically."

Second attempt Thought: Use a more specific query Action: [Search] "a specific query" Observation: Accurate results -- success

Implementation pattern

The react_loop below is unexecuted illustrative pseudocode. The max_iterations parameter caps iterations to prevent an infinite loop. The default of 5 is an example; the appropriate limit requires task-specific evaluation.

On each iteration, the LLM generates a Thought and decides an Action. The corresponding tool ([Search], [Calculate], and so on) is then executed. The resulting Observation accumulates in context, and once the goal is reached the final answer is returned.

def react_loop(question, max_iterations=5):
    context = f"Question: {question}\n"
 
    for i in range(max_iterations):
        # Generate a Thought
        thought = llm.generate(context + "Thought:")
        context += f"Thought: {thought}\n"
 
        # Decide an Action
        action = llm.generate(context + "Action:")
 
        if action.startswith("[Search]"):
            result = search(action.query)
        elif action.startswith("[Calculate]"):
            result = calculate(action.expression)
        else:
            break
 
        context += f"Action: {action}\n"
        context += f"Observation: {result}\n"
 
    # Generate the final answer
    answer = llm.generate(context + "Answer:")
    return answer

Related concepts

• Chain of Thought (CoT): the underlying technique (reasoning only)
• Reflexion: adds failure recovery
• LATS: adds tree search
• Toolformer: learns tool use automatically

Summary

ReAct is a pattern that cycles through Thought, Action, and Observation and reinforces reasoning with external tools. It supplies real-time information and fact verification that reasoning alone cannot provide. Visible tool use makes the process easy to verify.

It does fail as tool call sequences get complicated, and recovering from a wrong path is hard. Whether the differences in the original tasks persist on new tasks requires separate evaluation. Reinforce it by combining with Reflexion when failure recovery is needed, or with LATS when search is needed.