AI BENCHNOTES
Coding agentsBENCH NOTE 008HarnessesMCP
AI BENCHNOTES
THE COMPLETE WORKBENCHPick a machine part.
12 NOTES / 12 LABS / ZERO INCENSE

Start anywhere. The numbers are a useful route, not homework police.

00Animated bench map
CONCEPT 008FIELD RATING

A model is a brain.The harness gives it a shift.

The harness is the runtime wrapped around a model: it assembles context, runs the tool loop, enforces permissions, remembers the thread, and decides when the work is actually done.

  • MODELchooses the next move
  • HARNESSmakes moves executable
  • CODING AGENTdoes software work through the rig
01 / THE SHORT VERSION

The harness is the part that turns suggestions into a controlled process.

People often call the whole product “the agent.” Useful shorthand, fuzzy engineering. Separating model, harness, and job-specific agent makes failures easier to locate—and systems easier to improve.

01
THE BRAIN

Model

Interprets the prompt and proposes text or tool calls. It does not, by itself, own a filesystem, persist a thread, or enforce your approval rules.

probabilistic
02
THE RIG

Harness

Builds each model input, routes tool calls, returns observations, manages context and session state, and applies runtime policy.

software
03
THE JOB

Coding agent

A model operating through a software-shaped harness with repository tools, instructions, tests, and a user experience for engineering work.

system

A stronger model can improve the driver. A better harness improves the road, dashboard, brakes, map, memory, and rules the driver operates inside.

02 / LOAD-BEARING LAB

Same brain. Different shift. More rig.

Choose the job. Watch the runtime grow around the same model as the task becomes longer, riskier, and harder to keep in one context.

MISSION / REFACTOR MEDIUM BLAST RADIUS
USER INTENT

Move token parsing to the boundary without changing behavior.

MODELSAME CORE
TIME HORIZON45 MIN
HARNESS MODULES
contexttoolssandboxapprovalstestsmemory + checkpointstracingstop policy
  1. 01
    maptrace callers + invariants
  2. 02
    patchedit multiple modules
  3. 03
    testobserve failure
  4. 04
    recoverrevise the patch
  5. 05
    handoffdiff + checks + caveats
WHAT CHANGED?

The loop is only half the story. Policy, context, recovery, and evidence keep the refactor bounded.

7 load-bearing modules
03 / NAME THE LAYERS

A harness is not a coding agent. A coding agent usually contains one.

Product language blurs this boundary constantly. That is fine in a demo. During debugging, decide whether the mistake came from the model’s choice, the harness’s execution, or the coding workflow’s definition of success.

MODELjudgment
HARNESSruntime
SOFTWARE WORKFLOWtools + rules + checks
CODING AGENTbounded engineering work
QUESTIONHARNESSCODING AGENT
What is it?

Runtime and orchestration software around one or more model calls.

A system or product that performs software-engineering tasks.

What does it own?

Loop, context assembly, sessions, tool routing, permissions, events, stop conditions.

The job: inspect the repo, plan, edit, test, recover, and report.

Is it coding-specific?

Not necessarily. A research, support, or browser agent also needs a harness.

Yes. Its tools, instructions, feedback, and success criteria are shaped around code.

Can you swap the model?

Often. A good boundary lets the runtime evolve separately from the model.

Yes, but behavior still changes because the model is one major component of the full system.

BENCH NOTE 007Now inspect the worker, not the rig.
04 / UNDER THE BUCKLES

Six jobs commonly hiding inside “call the model again.”

Implementations divide these responsibilities differently. The durable point is that all six are software behavior around the model—not secret intelligence inside it.

01

Input builder

Combines system instructions, repository guidance, user intent, available tools, prior events, and relevant environment facts.

Context is selected, not dumped.
02

Loop controller

Calls the model, distinguishes a final answer from a tool request, executes allowed actions, appends observations, and repeats.

A loop needs a budget and an exit.
03

Tool router

Validates arguments, maps a requested tool to real infrastructure, normalizes results, and returns errors the model can act on.

A schema is not authorization.
04

Policy boundary

Applies sandbox scope, network rules, approvals, credential handling, and other constraints before side effects occur.

Enforce outside the prompt.
05

State carrier

Persists threads and artifacts, compacts context, restores work, and keeps long tasks from depending on one model window.

The transcript is not the whole state.
06

Event recorder

Emits typed progress, tool calls, approvals, diffs, errors, usage, and traces so clients and operators can inspect the run.

If you cannot see it, you cannot tune it.
05 / RIGGING RULES

Good harnesses make the right thing easy to inspect.

Do not turn every model limitation into permanent orchestration. Harness assumptions age. Re-test which scaffolding is still load-bearing as models and tools improve.

  1. 01

    Make the environment legible.

    Put durable facts in inspectable files, schemas, commands, logs, and tests. Invisible tribal knowledge cannot steer a run.

  2. 02

    Keep policy out of the model’s imagination.

    Prompts explain. Sandboxes, validators, approval gates, and credential scopes enforce.

  3. 03

    Preserve truth outside the context window.

    For long work, persist plans, checkpoints, test state, and handoff notes where a fresh session can recover them.

  4. 04

    Expose one clean action surface.

    Tools should have sharp names, typed arguments, bounded outputs, actionable errors, and the least authority required.

  5. 05

    Design the stop as carefully as the loop.

    Define success, budgets, retry limits, escalation conditions, cancellation, and what evidence a final handoff must contain.

06 / FAILURE POSTERS

The rig can fail while the brain looks clever.

If a run goes wrong, inspect the boundary before blaming model intelligence. Many “agent failures” are stale context, permissive tools, hidden state, or weak stopping logic.

FAULT 01

The golden-cage harness

Fifty rigid steps make a capable model behave like a brittle macro.

BENCH FIX

Encode invariants and interfaces; leave local judgment to the model.

FAULT 02

Prompt-only seatbelts

The model is politely asked not to do something the runtime still allows.

BENCH FIX

Move hard boundaries into executable policy, sandboxing, and approvals.

FAULT 03

Transcript amnesia

A compacted or resumed run forgets what was changed, tested, or still broken.

BENCH FIX

Persist task truth in artifacts that survive the conversation window.

FAULT 04

Infinite useful motion

The loop keeps finding plausible work because “done” was never operationalized.

BENCH FIX

Add budgets, explicit success checks, escalation, and a final evidence contract.

07 / LOOP RECEIPT

The model proposes. The harness carries consequences.

This is illustrative pseudocode, not a universal SDK. Notice where the boundaries live: tools are filtered, arguments validated, actions authorized, execution sandboxed, events recorded, state checkpointed, and “done” verified outside the model.

harness.ts / conceptualMODEL-AGNOSTIC
while (budget.remaining()) {
  const input = context.build({
    thread,
    instructions,
    tools: policy.allowedTools(),
    workspace: await inspectEnvironment(),
  });

  const move = await model.respond(input);
  events.record(move);

  if (move.type === "final") {
    return verifyHandoff(move, workspace);
  }

  const call = validate(move.toolCall);
  await policy.authorize(call);

  const observation = await sandbox.execute(call);
  thread.append(observation);

  if (context.needsCompaction()) {
    await checkpoint.persist(thread, workspace);
    thread = await context.compact(thread);
  }
}

return escalate("Budget exhausted with unfinished work");
KEEP THISbrain ≠ rig ≠ job

Improve each layer separately. Evaluate the behavior of the whole system together.