AI BENCHNOTES
Agent loopsCHAPTER 007Coding agentsHarnesses
AI BENCHNOTES
PLAIN-LANGUAGE VIEWChoose a question.
12 CHAPTERS / EVERYDAY EXAMPLES / NO CODE REQUIRED

Start with a familiar question. Each chapter introduces the standard technical term.

00Plain-language topic guide
CHAPTER 007repository tasks

Coding agentsinspect, edit,and verify.

Coding agents:change codeand show their checks.

A coding agent inspects a repository, forms a bounded plan, uses tools, reads the results, revises its work, and returns the final diff with evidence.A coding agent can explore a software project, make a focused change, run checks, and revise when those checks disagree. Its job is to return the change plus a reason to trust it.

  • SEARCHLOOK FIRSTbefore changing
  • PATCHCHANGE LESSthe smallest surfaceonly what the task needs
  • PROVECHECKbefore handing back
01 / OVERVIEW

Code completion predicts text. A coding agent uses repository feedback.Autocomplete finishes a line. An agent works through a checked task.

A coding agent closes a feedback loop: observe the repository, make a bounded change, run a check, and revise from the result.A coding agent looks around, makes a small change, runs a check that can disagree, and revises when the check fails.

CODE COMPLETION

Current code context goes in.It predicts what comes next.

const total = items.reduce(…)A likely next line appears.

A plausible continuation comes out. Useful—and then it stops.That suggestion may be useful, but it does not inspect or test the rest of the project.

NO FEEDBACK LOOP
CODING AGENT

A task is evaluated against the repository.A task meets the whole project.

  1. 01Read rules + current stateRead the project and its rules
  2. 02Search for the real change surfaceFind the files that actually matter
  3. 03Patch, execute, inspect, reviseChange, test, inspect, revise
  4. 04Return the diff + evidence + caveatsShow what changed, the checks, and any uncertainty
CHECKS PROVIDE FEEDBACK

A useful handoff includes the diff, the checks that passed, and any remaining uncertainty.A useful result includes the change, the checks that passed, and anything still uncertain.

02 / INTERACTIVE EXAMPLE

Repository evidence changes the quality of the patch.

Same task. Same model. One control changes how much repository evidence the loop may collect before it returns a result.

EXAMPLE TASKAdd owner_name to the CSV export.

Keep column order stable. Update relevant tests. Do not edit generated output or unrelated dirty files.

TASK CSV-42

Lower settings remove observations and checks that can expose a wrong edit.

  1. 01
    git status + AGENTS.md

    Read instructions and current state

    Find local rules, allowed files, and three unrelated user edits.

  2. 02
    rg "owner|csv"

    Locate the source

    Search connects the schema, exporter, and targeted test.

  3. 03
    src/export/csv.ts

    Edit the smallest source change

    Change the header and source serializer; leave nearby cleanup alone.

  4. 04
    pnpm test csv

    Run a focused check

    FAIL: row serializer omitted owner. Revise the patch. PASS.

  5. 05
    typecheck + git diff

    Inspect the final diff

    Types pass. Diff matches scope. Hand back checks and caveats.

HANDOFF READY
The change arrives with evidence.

Targeted tests and typecheck pass; the final diff is inspected; pre-existing dirty files stay untouched.

03 / CONTROL LOOP

Tool results determine the agent’s next step.

Models propose the next action. The runtime supplies tools, permissions, files, command output, and stopping conditions. Each observation changes what action makes sense next.

CODING AGENT CONTROL LOOPCHECK RESULTS GUIDE THE NEXT ACTION
  1. 01
    INSPECT

    Read project instructions and current state.

    Load the task, nearest repository instructions, current worktree state, and relevant project map.

  2. 02
    LOCATE

    Trace the change surface.

    Find names, call sites, tests, and ownership before deciding which files to edit.

  3. 03
    BOUND

    Set boundaries before editing.

    Name the files, constraints, expected behavior, and checks. Ask before crossing a permission boundary.

  4. 04
    PATCH

    Edit the source of truth.

    Edit in reviewable increments. Generated files, build output, and vendored code usually have a source upstream.

  5. 05
    EXERCISE

    Run the closest meaningful check.

    Start with a targeted test, typecheck, lint, or build that can expose the specific mistake quickly.

  6. 06
    HAND BACK

    Return the diff and its evidence.

    Inspect the diff. Report what changed, what passed, what was preserved, and what remains uncertain.

A failed check gives the loop specific information for the next revision.

04 / TASK DEFINITION

Define the outcome, boundaries, and completion checks.

The agent can discover implementation details. The task should provide the desired outcome, acceptable scope, and definition of done.

VAGUE TASKNO. ???
“Improve CSV exports.”
  • No observable result
  • No boundary around adjacent cleanup
  • No check that defines “improved”
INSUFFICIENTLY DEFINED
BOUNDED TASKNO. CSV-42
OUTCOME

CSV exports include owner_name after project_name.

ACCEPTANCE

Existing column order stays stable; blank owners serialize as empty.

SCOPE

Exporter source + focused tests. Preserve unrelated dirty files.

BOUNDARY

Do not edit generated clients or change the public schema.

EVIDENCE

Run export tests and TypeScript checks; report anything not run.

AGENT-COMPLETABLE

Let the agent discover where the change belongs. Specify what success means.

05 / OPERATING RULES

Six rules for safe, reviewable repository work.

OBSERVE→ PATCH SMALL→ CHECK→ WIDEN IF NEEDED
  1. 01READ

    Read the nearest instructions.

    Repository guidance, directory-specific rules, conventions, and requested commands define the local constraints.

  2. 02STATUS

    Take a worktree baseline.

    Check status before editing. Pre-existing modifications belong to their author; preserve them and avoid accidental bundling.

  3. 03TRACE

    Search through the behavior.

    Find definitions, references, tests, schemas, and generators. The first matching file is a clue, not permission to edit.

  4. 04SMALL

    Patch at checkpoint size.

    Make one coherent change, inspect the diff, run the closest check, then widen only when evidence requires it.

  5. 05ASK

    Respect sandbox and approval boundaries.

    Sandboxes and approval gates are part of the system. If a command needs more access, explain why and request it—do not route around it.

  6. 06PROVE

    Verify in proportion to risk.

    A copy change may need a build. A data migration needs considerably more. “Looks right” is not an executable check.

CHECKPOINT RHYTHM
  1. observe
  2. edit one idea
  3. inspect diff
  4. run closest check
  5. record evidence
06 / FAILURE MODES

Six recurring coding-agent failures.

Most coding-agent failures are ordinary software-process mistakes performed quickly and at scale.

ISSUE 01

Editing before searching

Behavior: Opens the obvious file and edits immediately.

Correction: Search references, tests, and ownership before choosing the change surface.

ISSUE 02

Editing generated output

Behavior: Fixes dist/, a generated client, lock output, or compiled assets.

Correction: Find the generator or source-of-truth; regenerate only when the workflow calls for it.

ISSUE 03

Overwriting existing work

Behavior: Treats every existing diff as agent-created—or disposable.

Correction: Baseline status, isolate the requested patch, and name what was intentionally preserved.

ISSUE 04

Expanding the scope

Behavior: Mixes the requested fix with refactoring, formatting, and unrelated cleanup.

Correction: Use small coherent patches. Review and verify before expanding scope.

ISSUE 05

Bypassing permissions

Behavior: Works around a sandbox, secret, network, or approval boundary.

Correction: Pause, state the blocked action and reason, then request the narrow authority needed.

ISSUE 06

Claiming success without checks

Behavior: Reports success because the edit looks plausible.

Correction: Run the nearest meaningful check, inspect its output, and disclose checks that could not run.

07 / IMPLEMENTATION

The model chooses. The runtime makes it real.

“Agent” describes a system, not just a model call. The surrounding runtime decides which files and tools are visible, executes actions, returns observations, enforces permissions, and stops the loop.

PROVIDER-NEUTRAL PSEUDOCODErepair-loop.txt
function repair(task, repo, policy):
  rules    = read_nearest_guidance(repo)
  baseline = inspect_status(repo)
  evidence = search(task, repo)
  plan     = bound_change(task, rules, evidence)

  while plan.has_next_step():
    require_permission(plan.next_step, policy)
    patch  = edit_smallest_source(plan)
    result = run_closest_check(patch)

    if result.failed:
      plan = revise_from(result)
      continue

    return handoff(
      diff=inspect_diff(baseline),
      checks=result.passed,
      caveats=result.not_run
    )

  return blocked(reason, next_evidence_needed)
08 / MISCONCEPTIONS

Four common misconceptions about coding agents.

01

“It reads the whole repository.”

Usually it sees selected files, search results, instructions, diffs, and command output inside a bounded context. Good retrieval and inspection still matter.

02

“More autonomy means fewer checkpoints.”

Longer loops create more opportunities to drift. Small patches and explicit checkpoints make recovery cheaper.

03

“Passing tests proves correctness.”

Tests prove only the expectations they encode. Review the diff, consider missing coverage, and state caveats.

04

“The fastest agent makes the biggest patch.”

A smaller change is easier to inspect, test, revert, and trust. Large patches make review and recovery more expensive.

AI BENCHNOTES

For builders who want clear mental models and implementation detail.For curious people who want to understand how AI products work.

Back to top ↑