CHAPTER 006multi-step control

Agent loops repeatdecisions, actions,and observations.

Agent loops:take one step,check it, then continue.

Your application asks a model what to do next, validates and runs the requested tool, feeds the result back, and decides whether another turn is allowed. The application owns the loop and its limits.An AI agent looks at the situation, chooses one next step, checks what happened, and then decides whether to continue. The surrounding app controls the tools, limits, and stop button.

Try the exit-policy example Show me the loop Show me the safety controls

DISCRETESTEPWISEmodel callsone move at a time
STATEFULKEEPS NOTESevent historyresults shape the next move
BOUNDEDLIMITEDby your codeby rules and stop conditions

AGENT LOOP / CONTROL FLOW ORCHESTRATOR ON

01Observestate

02Decidenext move

03Actthrough tool

04Inspectresult

APPLICATIONHARNESScontrols each turn

AFTER INSPECTIONSTOP?yes → exitno ↺ next turn

The model chooses a move. The harness decides whether that move is permitted, executed, recorded, and repeated.

01 / OVERVIEW

Each model call is one step in an application-controlled loop.The AI is called for one step, then the app checks the result.

A model invocation ends when it returns. Your application may then run code for milliseconds, minutes, or hours before calling the model again. Saved state and history connect the steps.The AI is not thinking continuously in the background. The app saves what happened and calls it again only when another step is needed.

STEP 01HARNESSAPP
Observe
Pack the goal, current state, relevant history, and fresh evidence.Gather the goal, what has happened so far, and any new information.
STEP 02MODELAI
Decide
Return a final answer or a structured request for the next tool.Give a final answer or ask to use one tool for the next step.
STEP 03APP + TOOLAPP + TOOL
Act
Validate, approve if needed, then execute one bounded operation.Check the request, ask for approval if needed, then do one limited action.
STEP 04HARNESSAPP
Inspect
Record the result, errors, and any environment changes as new state.Save what worked, what failed, and what changed.
STEP 05POLICYRULES
Stop or continue
Check proof of success and budgets. Exit—or make another model call.Check whether the goal is proven complete and whether time or cost limits remain.

Between model calls: the runtime executes code, waits for tools, persists state, or pauses for a person.Between steps: the app is doing ordinary work: waiting for a tool, saving progress, or pausing for a person.

02 / INTERACTIVE EXAMPLE

The stop condition changes the outcome.

Same goal. Same tools. Same first error. Change only what the harness considers a reason to stop. The run will recover, stop early, or exhaust its turn budget.

Choose one exit policy

Proof of doneRecover from a retryable tool error; exit only after tests pass.First snagTreat any tool failure as terminal, even when recovery is safe.No finish lineNo external proof of success; the turn budget is the final limit.

RUN / FIX_FLAKY_AUTH_TESTEXIT POLICY: PROOF OF DONE

T01OBSERVEOne failing auth test
T02DECIDEPatch the stale assertion
T02TOOLPatch conflict: line moved
T03INSPECTRetryable; no change applied
T04ACTRe-read, patch once, run tests
T05VERIFY42 tests pass → exit

RUN RESULTDONE5 turns · recovered

The loop treated the stale patch as evidence, retried safely, and exited when the environment proved the task was complete.

HARD LIMITSmax turns: 8tool retries: 1side effects: approval gated

Define completion with an observable check, and add a budget for runs that never satisfy it.

03 / ROLES AND STATE

Reliable loops separate judgment, orchestration, and execution.Reliable agents keep choosing, controlling, and doing separate.

The model proposes the next step. The application harness applies policy and carries state. Tools execute actions and report what happened.A reliable agent separates choosing, controlling, and doing. The AI suggests. The app checks. Tools touch the real world.

01 / APPLICATION HARNESS01 / THE APPControls the loop

Builds each model input, validates tool calls, carries state, enforces permissions and budgets, records the trace, and decides whether another turn happens.Gathers the reading material, checks tool requests, saves progress, enforces permissions and limits, and decides whether another step may happen.

02 / MODEL TURN02 / THE AIThe next-move proposerThe next-step suggester

Reads the current evidence and returns a bounded next action, perhaps a plan update, a tool request, a question for a human, or a candidate final answer.Reads what has happened so far and suggests one next step: update the plan, ask for a tool, ask a person, or give a final answer.

03 / TOOLS + ENVIRONMENT03 / TOOLS + REAL WORLDExecutes actions and reports results

Search, read, write, test, transact, or inspect. Their results reveal what actually happened—not what the model hoped happened.Tools search, read, change, test, buy, or inspect. Their results show what actually happened—not what the AI hoped happened.

RE-INSPECT AFTER ACTIONS

After an action, inspect the environment again. A successful API response can still produce the wrong state; a failed tool call can contain exactly the evidence needed to recover.After an action, look at the real world again. A tool can report success while leaving the wrong result, and a failed tool can still provide the clue needed to recover.

04 / CONTROL RULES

Bound the loop with completion checks, budgets, and safe retries.

Most loop failures are ordinary software failures: missing termination, unsafe retries, stale state, weak permissions, or an incomplete trace.

01
TERMINATIONWrite an exit predicate
Completion should be observable: tests pass, required fields exist, a cited answer meets a rubric, or the user confirms. “The model sounded finished” is not a predicate.
02
BUDGETSSet hard limits
Cap turns, elapsed time, tokens or spend, tool calls, and repeated failures. Treat a budget stop as a defined outcome.
03
RECOVERYClassify errors before retrying
Retry transient failures with backoff. Feed actionable failures back as observations. Stop or escalate permission errors, invalid requests, and repeated identical failures.
04
IDEMPOTENCYMake repetition safe
Use idempotency keys, read-before-write checks, operation IDs, or deduplication. A timed-out payment call is not permission to charge twice.
05
HUMAN GATERequire approval for consequential actions
Ask for approval before irreversible, expensive, public, privileged, or ambiguous actions. Persist the run so approval can arrive without replaying completed work.
06
STATEKeep a useful trace
Record decisions, tool inputs and results, retries, approvals, and stop reasons. Compact old history deliberately; do not let an ever-growing transcript become the control system.

PLAN-AHEAD MODEExplicit milestones

Useful when dependencies matter: outline milestones, reserve resources, and make the intended path inspectable.

REACTIVE MODEFrequent re-evaluation

Useful when the environment is uncertain: take a small action, inspect reality, then choose again.

Treat plans as revisable state. Tool results should change the plan when the environment differs from the model’s assumptions.

05 / WHEN TO USE A LOOP

Use a loop when the next step depends on observed results.

A loop earns its complexity when each observation can change the next action. Use a fixed workflow when the path is already known.

USE A LOOPuncertain path · observable progress

Repository work
The path depends on what search, edits, tests, and code review reveal.
Open-ended research
The next query depends on gaps in the evidence, with a clear coverage rubric.
Incident triage
Inspect signals, form a hypothesis, run a safe diagnostic, and update.
Messy operations
Reconcile records or complete forms where each result changes the next step.

USE A FIXED WORKFLOWknown path · predictable steps

The path is already known
Use normal deterministic code or a fixed workflow. A loop adds latency and new failure modes.
Success cannot be observed
Without evidence or a rubric, the loop cannot know whether to continue, stop, or correct itself.
Every action is irreversible
Redesign tools and approval boundaries before granting repeated autonomous access.
One response solves it
Do not build a tiny bureaucracy around a question that needs one model call.

DESIGN RULE
Give the model freedom over the path, not freedom from constraints.

06 / IMPLEMENTATION

A controlled loop alternates model calls with validated execution.

Frameworks package this control flow differently. The durable responsibilities are to inspect, decide, validate, execute, record, verify, and stop.

agent-loop.ts · provider-neutral pseudocodeCONTROL LIVES HERE

async function runAgent(goal: Goal, budget: Budget) {
  const state = await loadRunState(goal);

  while (state.turns < budget.maxTurns) {
    const observation = await inspectEnvironment(state);
    state.record({ type: "observation", observation });

    if (completionPolicy.isSatisfied(goal, observation)) {
      return finish(state, "verified_complete");
    }

    state.turns += 1; // one turn = one new model invocation
    const next = await model.decide({
      goal,
      state: state.compactView(),
      observation,
      tools: allowedTools,
    });

    if (next.kind === "ask_human") {
      return pauseForApproval(state, next.question);
    }

    if (next.kind === "final") {
      state.record({ type: "candidate_final", text: next.text });
      await saveRunState(state);
      continue; // verify against the environment on the next cycle
    }

    if (approvalPolicy.requiresHuman(next.toolCall)) {
      return pauseForApproval(state, next.toolCall);
    }

    const result = await executeTool(next.toolCall, {
      idempotencyKey: state.operationId(next.toolCall),
      retry: retryPolicy.for(next.toolCall),
    });

    state.record({ type: "tool_result", result });
    await saveRunState(state);
  }

  return finish(state, "turn_budget_exhausted");
}

WHY INSPECT FIRST?

The environment may already satisfy the goal after a resume, duplicate delivery, human edit, or delayed tool completion.

WHY RECORD BEFORE NEXT TURN?

Recovery needs durable facts. If the process restarts, the loop should continue from state—not replay side effects from memory.

WHY VERIFY “FINAL”?

Final text is a proposal to stop. When ground truth is available, let code or a grader confirm the claim.

07 / FAILURE MODES

Six ways agent loops fail.

“The model keeps thinking between tool calls.”

Usually, the application is executing a tool or waiting. A later model call receives the accumulated state and continues from that evidence.

“More turns make the agent smarter.”

More turns buy opportunities to observe and correct. They also add cost, latency, and chances for errors to compound.

“Retry means run the same thing again.”

Only retry classified transient failures, and make side effects idempotent. Otherwise repetition may duplicate damage.

“A detailed plan removes the need to inspect.”

Plans are hypotheses. Tool results and environment state outrank the plan whenever reality disagrees.

“If it says it is done, the loop is done.”

Treat self-reported completion as a candidate. Prefer tests, state checks, graders, or human confirmation when available.

“Human approval means clicking yes on every turn.”

Approval is most useful at meaningful boundaries: irreversible, privileged, expensive, public, or intent-sensitive actions.

SUMMARY

agent loop = model judgment + tool-grounded feedback + application control

↻ until verified done, paused, or safely out of budget