AI BENCHNOTES
EvalsBENCH NOTE 011GuardrailsTracing
AI BENCHNOTES
THE COMPLETE WORKBENCHPick a machine part.
12 NOTES / 12 LABS / ZERO INCENSE

Start anywhere. The numbers are a useful route, not homework police.

00Animated bench map
CONCEPT 011operationally spicy

Guardrails arethe ride controls,not a force field.

Models are uncertain components. Guardrails surround them with policy, permissions, checks, limits, and graceful exits—so a surprising answer does not automatically become a surprising incident.

The model may propose. The system still decides.

BENCH NOTE AMUSEMENTS PRESENTSMAYBE MOUNTAIN

an uncertain ride with extremely certain brakes

Ppolicy says whatEcode enforces itDsignals watch it
FAILURE
STILL POSSIBLE
01 / THE SHORT VERSION

A guardrail is a control system, not a stern paragraph.

The model is one part of the product. The product owns the consequences. Separate these three jobs or your safety story becomes one very optimistic prompt.

Policy

Decide what should happen.

Write the boundaries: which content, actions, users, amounts, and situations are allowed, denied, or require review.

Refunds above $50 require an operator.
Enforcement

Make code hold the line.

Use schemas, permissions, deterministic rules, sandboxes, and approval gates. Enforcement should not depend on the model agreeing.

The payment function rejects amount > 50.
Detection

Notice what slipped, drifted, or spiked.

Classifiers, anomaly signals, traces, audits, and evals reveal suspicious behavior. Detection is evidence—not a magic verdict.

Refund attempts spike 8× in ten minutes.
02 / MAYBE MOUNTAIN CONTROL BOOTH

Same model. Different stakes. Different ride plan.

Choose the task on the ticket. Watch the product—not the model—decide which controls matter and where the request leaves the ride.

Choose an AI system task
INCOMING RIDE REQUEST

“Where is order #214? Give me the tracking link.”

STAKESread-only · reversible
  1. 01
    POLICY

    Entrance policy

    Order support is an allowed use case.

    IN SCOPE
  2. 02
    POLICY

    Instruction hierarchy

    Pasted text cannot rewrite product rules.

    SYSTEM WINS
  3. 03
    ENFORCE

    Output schema

    Only summary and trackingUrl are accepted.

    TWO FIELDS
  4. 04
    ENFORCE

    Tool wristband

    orders.read is scoped to this user.

    READ ONLY
  5. 05
    ENFORCE

    Business rules

    Code verifies the order belongs to the caller.

    OWNER MATCH
  6. 06
    ENFORCE

    Exit inspection

    Escape text and allow-list the carrier domain.

    SANITIZE
  7. 07
    DETECT

    Ride recorder

    Log the decision, retrieval, and final response.

    TRACE
RIDE OPERATOR SAYSALLOW + SHAPE

Return a plain-text summary and an allow-listed carrier link.

The request is ordinary, but ownership, data exposure, and unsafe links are still system concerns.

RESIDUAL RISKLow, not zeroControls reduce risk. They do not erase reality.

A classifier may help route the request, but deterministic permissions and business rules decide what can actually happen.

03 / RIDE ANATOMY

Nine places to install a brake.

You rarely need every control at maximum strength. You do need to know which failure each layer catches—and what happens when it catches one.

  1. BEFORE01

    Input policy

    Define supported use cases, prohibited content, identity requirements, and routes for sensitive requests.

    policy
  2. BEFORE02

    Instruction hierarchy

    Keep product rules in higher-priority instructions and treat user or retrieved text as untrusted data.

    policy
  3. PROPOSE03

    Structured output

    Make the model produce typed fields with narrow enums and required values. Parse; reject; retry safely.

    enforce
  4. ACT04

    Least-privilege tools

    Expose only the capabilities needed now, scoped to the current user, resource, and smallest useful action.

    enforce
  5. ACT05

    Business rules

    Enforce ownership, amount limits, state transitions, idempotency, and invariants in ordinary deterministic code.

    enforce
  6. ACT06

    Approval gates

    Require informed human confirmation for consequential, ambiguous, expensive, or hard-to-reverse actions.

    enforce
  7. AFTER07

    Output checks

    Validate, escape, redact, ground, or quarantine model output before another system or person consumes it.

    enforce
  8. ALWAYS08

    Rate & budget limits

    Cap calls, tokens, spend, retries, action frequency, and fan-out. A loop should have a fuse.

    enforce
  9. ALWAYS09

    Monitoring & fallback

    Trace decisions, watch drift, provide a useful refusal or handoff, and preserve enough evidence to improve.

    detect
GRACEFUL EXIT ≠ DEAD END

A useful refusal says what did not happen, why the product cannot do it, and what safe next step is available. “Computer says no” is not a recovery design.

04 / THE ANNOYING TRUTH

A tighter gate catches more trouble—and more innocent luggage.

Detection thresholds trade false negatives against false positives. The right setting depends on impact, reversibility, user context, and the quality of your fallback.

FALSE POSITIVESafe thing gets stopped.

User friction, lost utility, support tickets, unequal access.

FALSE NEGATIVERisky thing slips through.

Harm, loss, policy breach, data exposure, broken trust.

LOW IMPACTAllow, shape, observe.

Prefer recovery and good logs over gratuitous friction.

MEDIUM IMPACTVerify, limit, make reversible.

Bound permissions and add targeted confirmation.

HIGH IMPACTSeparate authority from generation.

Strong identity, deterministic gates, explicit approval.

Tune with representative evals and production evidence. A threshold chosen from three spicy demo prompts is performance art.

05 / OPERATOR MANUAL

Six rules for systems that can say “not like that.”

BOUND THEEFFECT.OBSERVE THEMISS.
  1. 01

    Start from the consequence.

    List what the system can expose, spend, change, send, or delete. Guard the effect, not merely the wording.

  2. 02

    Give the model a pencil, not the company stamp.

    Let it draft typed proposals. Keep authorization and execution in code with explicit identities and scopes.

  3. 03

    Use least privilege per turn.

    Offer the smallest tool set and narrowest credentials needed for the current task. Capability is blast radius.

  4. 04

    Make expensive actions boringly deterministic.

    Money, permissions, deletion, publication, and external messages deserve invariants, idempotency, and approvals.

  5. 05

    Design the refusal and the recovery together.

    Explain the boundary, preserve user work, and offer a safe route forward. A block without a next step trains bypass attempts.

  6. 06

    Test layers separately—and together.

    Measure classifier misses, schema failures, permission escapes, approval quality, and end-to-end outcomes.

06 / IMPLEMENTATION RECEIPT

The prompt is the memo. The code is the lock.

A provider-neutral sketch of a refund path. Exact APIs differ; the separation of proposal, authorization, execution, and audit is the durable part.

refund-ride.tsPSEUDO-TYPESCRIPT
// Illustrative TypeScript: the model proposes; code disposes.
async function handleRefund(input: Request, actor: User) {
  const route = await detectRisk(input);
  if (route === "unsupported") return safeFallback();

  const raw = await model.generate({
    instructions: REFUND_POLICY,
    input,
    tools: [{ name: "propose_refund" }], // no payment credential
    outputSchema: RefundProposal,
  });

  const proposal = parseWithSchema(raw, RefundProposal);
  const order = await orders.readForUser(proposal.orderId, actor.id);

  // Deterministic policy: never delegated to the model.
  assertRefundable(order);
  assertCurrency(proposal.amount, order.currency);
  assertWithinLimit(proposal.amount, 50);
  assertDailyBudget(actor.id);

  const approval = await approvals.request({
    actor,
    proposal,
    evidence: { deliveryState: order.deliveryState },
  });
  if (!approval.granted) return explainNoAction();

  const result = await payments.refund({
    orderId: order.id,
    amount: proposal.amount,
    idempotencyKey: approval.id,
  });

  await audit.record({ actor, proposal, approval, result });
  return sanitizeReceipt(result);
}
  1. 01

    Detection routes; it does not authorize.

    A risk signal can block or escalate, but a “clean” score never grants permission by itself.

  2. 02

    The model gets proposal-shaped tools.

    Its output is typed data. The payment credential stays in the deterministic execution layer.

  3. 03

    Rules run after parsing.

    Ownership, amount, budget, state, and currency are checked in code against live system data.

  4. 04

    Approval is bound to evidence.

    The operator sees the exact action and facts, not a cheerful one-line “looks safe” summary.

07 / INSPECTION REPORT

Six guardrails that mostly guard your feelings.

The common failure is not “no safety feature.” It is asking one layer to do a job it cannot actually perform.

01THE VELVET ROPE

A prompt-only perimeter

“Never do anything unsafe” is useful instruction, not an access-control system. It cannot revoke credentials or enforce a transaction invariant.

Back it with code, scope, and approvals.
02THE ORACLE

One classifier rules all

Detectors make errors and drift across languages, domains, and attacks. A score is a noisy measurement, not ground truth.

Calibrate, combine signals, and preserve appeals.
03THE MASTER KEY

One enormous tool

A support task needs read_order; it does not need admin_everything. Broad tools turn one model mistake into an infrastructure event.

Split capabilities and scope credentials.
04THE CONFETTI BUTTON

Approval without information

Humans rubber-stamp vague dialogs. “The AI wants to continue—allow?” is ceremony, not oversight.

Show the exact action, target, evidence, and undo.
05THE QUIET EJECTOR

Blocking without recovery

Opaque refusals strand legitimate users and encourage prompt gymnastics. They also hide which control fired.

State the boundary and offer a safe next step.
06THE LOG ATTIC

Tracing everything, learning nothing

A pile of sensitive prompts is not observability. Without ownership, alerts, review, retention, and eval feedback, logs become liability.

Record decisions deliberately and close the loop.
08 / LOST & FOUND

Three myths left near the exit.

“Guardrails make the system safe.”

They reduce specific risks under specific assumptions. You still need threat modeling, evals, monitoring, incident response, and sane product scope.

“Structured output means correct output.”

A schema proves shape, not truth, authority, freshness, or policy compliance. Valid JSON can still request a terrible refund.

“A human in the loop solves it.”

Only if the human has context, time, authority, and a usable decision surface. Otherwise you built a latency-shaped rubber stamp.

TAKEAWAY / KEEP THIS STUBGuardrails are layered controls around uncertainty—not evidence that uncertainty disappeared.

Define the boundary. Enforce the consequence. Detect the miss. Make the safe exit usable. Then keep testing the whole ride.