Agent Harnesses and Loop Engineering

Agent reliability comes less from how clever a single model call is than from the design of the runtime loop that wraps that call.

Three senses of harness

Harness is not a fixed term. Its meaning shifts with what it is wrapping. Recent literature uses the same word for at least three different things, and without separating them first, every paper becomes confusing.

The word originally comes from the test harness of software engineering. In the LLM world it spread first as the evaluation harness, as in lm-evaluation-harness around 2022 and 2023. From 2024 the meaning widened to the agent harness, the runtime layer wrapped around model calls.

This post is about the third one. Everything that happens after the first prompt belongs here. It is the code layer that manages the exchange between the model and its tools, filesystem and environment.

The four jobs of an agent harness

An agent harness handles four things: tool execution, context management, state persistence, and safety and verification. Pulling them apart makes it clear that as tasks get longer, execution reliability is constrained more by harness infrastructure than by model capability.

The diagram shows the order in which one turn runs. The model picks an action, the harness executes the tool, and the result is compacted before it goes back into the context. Progress is then written outside the session, safety invariants and stop conditions are checked, and control moves to the next turn. The model is called many times inside this loop, and the harness is what connects one call to the next.

Of the four jobs, context management ties directly to the token budget. Compaction keeps only what is needed from retrieval and tool output. The two basic axes are extraction, which preserves the original text, and summarization, which reconstructs the meaning. Tasks where provenance matters default to extraction; tasks where length is the pressing problem default to summarization.

Where cost leaks when the loop gets long

The same harness costs a completely different amount on a one-turn task than on a fifty-turn one. A single-turn chatbot pays the retrieval and tool overhead exactly once, so it never becomes a problem. A long-running agent pays it at every loop step, and the cost compounds.

The dominant cost driver is input tokens. The accumulated context that goes back to the model on every turn is billed on every call. Industry estimates put a ten-iteration loop at 5 to 30 times the token consumption of a single pass. The same estimates attribute 80 to 90 percent of agent system cost to inference (estimated).

All three failure modes arise when the loop cannot stop itself. If the decision of when to end retrieval, how many times to call a tool, and which output to throw away is left entirely to the model, cost drifts in the leaking direction. That is why the stopping machinery has to live in the harness.

Stopping a tool-calling loop

The basic design principle for tool-calling RAG is to state a maximum call count and an explicit loop termination condition. When the model decides on its own when to call retrieval or a tool, failure shows up in both directions: not calling when it should, and calling excessively on every query. Both grow when the termination condition is weak.

# pseudo code: termination control for a tool-calling loop (illustrative)
MAX_STEPS = 12
step = 0
while step < MAX_STEPS:                    # upper bound on call count
    action = model.decide(context)         # decide the next action
    if action.is_final:                    # the model declares it is done
        break
    result = dispatch(action)              # dispatch and run the tool
    context = compact(context + result)    # compact the context
    if evaluator.met(goal, context):       # a separate evaluator judges the stop condition
        break
    step += 1

MAX_STEPS here is the last safety net against an infinite loop. evaluator.met is the slot where a small, fast, fresh model, rather than the model that did the work, judges the termination condition. This is the maker-checker principle: just as the person who enters a large transfer is separated from the person who approves it, the generator is separated from the verifier. An agent that wrote the code grades its own result generously.

Tool calling introduces a new failure axis, so it gets its own observability metrics. The common failures are three: choosing the wrong tool, malformed input, and misreading the result. In production, keep watching tool call count, failure rate and average loop length so you catch the moment any of the three spikes.

Tools with side effects need a permission guard independent of termination control. Tools that write to the outside world, such as sending email or writing files, run only after user confirmation. Expose only authorized tools to the model, and record every call in an audit log. The safety filter has to treat the tool-calling decision itself as an object of inspection, not just the answer text.

Wiring retrieval in as a tool

There are two ways to put Retrieval-Augmented Generation (RAG) into a harness. Either retrieval is a fixed stage in the pipeline, or it is exposed as a tool the model calls when it decides to. The second option moves control over RAG from the system to the model.

Exposed as a tool, it looks to the model like one more function with a signature such as search(query, k) → documents. Here the design of the tool schema drives quality. If the only argument is a query string, the model carries the entire burden of refining the query. Adding filter arguments such as date, domain and permissions lets the model use metadata filters directly. Too many arguments, though, and the model gets confused or tries to fill in unnecessary filters on every call.

With identical retrieval infrastructure, a different agent loop shifts the whole performance distribution. This mirrors the 'model x harness' matrix in coding benchmarks, where changing only the harness layer moves the score. Community reports describe the same benchmark score rising after only the harness layer was reworked, with the model left untouched.

There is also research that treats the harness code itself as the optimization target. That view defines the harness as 'the code that decides what information is stored, retrieved and presented'. It keeps feedback in its raw form rather than compressing it into a summary, and searches over harnesses.

Loop engineering, one layer up

Above prompts, context and the harness sits one more layer. Addy Osmani named it loop engineering: pulling the human out of the working position and designing the system that does the work instead. If the harness makes one turn run well, loop engineering puts that turn on a schedule and turns it into a repetition.

A turn consists of five moves: discovery, handoff, verification, persistence and scheduling. Verification is both the easiest one to cut corners on and the one you must not skip. A loop without a real check is just an agent nodding at itself.

Context and memory have to be distinguished here. Context is what the model sees in this round, and it disappears when the window is cleared. Memory is state that persists across rounds and across days, written outside the conversation in something like a markdown file or a board. Keeping a loop's progress record only in the context window means one refresh erases all of it.

Summary

An agent harness is the runtime loop wrapped around model calls, and it splits into four jobs: tool execution, context management, state persistence, and safety verification. The longer the task, the more reliability is constrained by harness design rather than by the model. A longer loop compounds input token cost while leaking through retrieval thrash, tool storms and context bloat. That makes a maximum call count and termination condition, maker-checker verification by a separate evaluator, and observability over tool call metrics part of the baseline design.

Exposing retrieval as a tool hands control to the model, so schema design drives quality. The same retrieval infrastructure produces a different performance distribution under a different harness. Loop engineering, one layer above, puts that turn on a schedule and makes it repeat. It only works when context and memory are kept separate and the verification move is not skipped.