Reflexion

Analyze task feedback in language, keep it in memory, and consult that lesson on the next attempt.

The Idea

Reflexion analyzes task feedback in language and stores it in memory for the next attempt. Feedback can be external or internally generated, and can use numerical scores or free-form language. The diagram and code below illustrate retries after failure.

flowchart LR
    A["Attempt"] --> E["Environment evaluation"]
    E --> D{"Success"}
    D -->|yes| F["Finish"]
    D -->|no| R["Reflect on failure"]
    R --> M["Store lesson in memory"]
    M --> A

How It Works

1. First attempt fails
2. Reflection: analyze "why did it fail?" in language
3. Store the lesson in memory: record the reflection as text
4. Second attempt: consult memory, revise, and evaluate the result again

Example

The code and memory below are unexecuted illustrative examples. Stored feedback need not be correct, and a later attempt need not succeed.

# [First attempt]
def book_hotel(name, dates, room_type):
    reservation = {
        "name": name,
        "dates": dates,
        "room": room_type
    }
    return confirm_booking(reservation)
 
# Result: RuntimeError - 'dates' must be a list

Reflection: "The dates variable needs type validation. The user may pass a string or another type."

Stored in memory

# [Second attempt] - with memory
def book_hotel(name, dates, room_type):
    # learned from memory: type validation needed
    if not isinstance(dates, list):
        raise ValueError("dates must be a list")
 
    reservation = {
        "name": name,
        "dates": dates,
        "room": room_type
    }
    return confirm_booking(reservation)
 
# Result: success

Key Properties

Performance

The NeurIPS 2023 Reflexion paper (arXiv:2303.11366) evaluates acting, reasoning, and code generation. Its HumanEval results include execution feedback and iterative revision. They must not be read as equivalent to one generation without feedback.

How It Differs From Plain Retrying

Retries without feedback can still succeed through sample diversity. Reflexion differs by adding language feedback derived from earlier results to the next attempt. Evaluate the effect with matched retry counts and total compute budgets.

Memory Structure

# Reflexion memory example
memory = [
    {
        "task": "implement the book_hotel function",
        "failure": "TypeError: dates must be a list",
        "reflection": "input type validation is needed",
        "solution": "add an isinstance() check"
    },
    {
        "task": "API call function",
        "failure": "ConnectionError",
        "reflection": "network error handling is needed",
        "solution": "add try-except and retry logic"
    }
]

Using the Memory

New task: "implement a payment API function"

Memory lookup: related experience "API call function", needs network error handling

Application: include try-except and retry logic from the start and succeed on the first attempt.

How It Differs From Self-Refine

The Loop

def reflexion_loop(task, max_attempts=3):
    memory = load_memory()
 
    for attempt in range(max_attempts):
        # attempt with memory as context
        result = execute_with_memory(task, memory)
 
        if result.success:
            return result
 
        # reflect on failure
        reflection = reflect_on_failure(
            task=task,
            error=result.error,
            code=result.code
        )
 
        # store in memory
        memory.append({
            "task": task,
            "failure": result.error,
            "reflection": reflection
        })
 
    return result

load_memory loads the lessons accumulated from earlier attempts, and execute_with_memory runs the task with that memory as context. On failure, reflect_on_failure takes the error and code and states the cause in language. memory.append stores that reflection for the next attempt.

Where It Applies

Limits

1. Feedback quality: inaccurate external or internal evaluation can mislead the next attempt
2. Memory retrieval quality: the relevant experience may not be found
3. Generalization: similar situations may not be recognized as similar

Related Concepts

• Self-Refine: continuous improvement (no failure required)
• CRITIC: verification with external tools
• ReAct: the underlying framework
• Self-Debug (ICLR 2024, arXiv 2304.05128): self-correction that reads code execution traces and debugs itself

Summary

Reflexion analyzes task feedback in language and stores it in text memory for the next attempt. It does not update model weights, so this update requires no retraining. Feedback can come from external or internal evaluation, and its accuracy affects the next attempt.

If relevant experience is not retrieved, use of the stored feedback is limited. Evaluate effects for the task, feedback signal, and memory usage. Comparisons with Self-Refine also require matched tasks and computation budgets.