AI BENCHNOTES
HarnessesBENCH NOTE 009MCPEvals
AI BENCHNOTES
THE COMPLETE WORKBENCHPick a machine part.
12 NOTES / 12 LABS / ZERO INCENSE

Start anywhere. The numbers are a useful route, not homework police.

00Animated bench map
CONCEPT 009protocol flavored

MCP:one socket,many weird machines.

Model Context Protocol gives AI applications a shared way to discover and use external context and tools. The plug shape is standard. What you plug in—and whether you trust it—is still your problem.

Patch a server Show me the trust boundary
  • 1:1client to server
  • JSON-RPCmessage layer
  • ≠ TRUSTpermission is separate
01 / THE SHORT VERSION

MCP standardizes the conversation, not the decision.

A host opens a dedicated client connection to each server. They agree on a protocol version and supported capabilities, then exchange typed requests, responses, and notifications. The host still decides what reaches the model and what is allowed to run.

01
HOSTRuns the workshop

The AI application the user touches. It manages the model, permissions, context, consent, and a fleet of clients.

One host, many connections.
02
CLIENTWorks one radio

A protocol component created by the host. It negotiates and maintains one dedicated session with one server.

One client, one server.
03
SERVEROffers a specialty

A local process or remote service exposing focused resources, prompts, tools, and other negotiated features.

Useful does not mean trusted.
02 / PATCH-BAY LAB

Change the server. Keep the socket.

Patch the host into one server. The connection ritual stays recognizable; the discovered features, authorization boundary, and request all change.

PATCH SESSION / ISSUES NEGOTIATED
HOSTAI WORKBENCH
USER INTENT

Turn this bug report into a tracked issue.

HOST KEEPSconversationmodelconsent
  1. 01
    initialize

    Agree on version; exchange capability categories.

  2. 02
    discover

    */list reveals the actual inventory.

  3. 03
    host gate

    Mutation ahead. The host shows the server, tool, and arguments to the user before sending.

  4. 04
    tools/call

    create_issue { title, body }

  5. 05
    response

    Issue #842 created · URL returned

REMOTE SERVICEStreamable HTTP
MCP SERVERIssue tracker
ADVERTISED CAPABILITIES
TRP
DISCOVERED INVENTORY
  • TOOLcreate_issue
  • RESOURCEproject://roadmap
  • PROMPTtriage_bug
AUTHORIZATION BOUNDARY

The client’s OAuth token is audience-bound to this MCP server. Any upstream tracker credential is a separate token.

action completedIssue #842 created · URL returned

Same protocol shape. Different capability set, credential path, and blast radius.

03 / UNDER THE DECK

The host has one hand on every plug.

“The model connects to MCP” is convenient shorthand, but it erases the useful boundary. The host creates clients, each client maintains one server session, and servers remain isolated from the full conversation and from one another.

SESSION LOGHandshake before horsepower.

Capability negotiation says which feature families are legal. Discovery methods such as tools/list reveal the actual items inside those families.

  1. 01
    INITinitialize

    The client proposes a protocol version, capabilities, and identity. The server replies with its supported version, capabilities, and identity.

  2. 02
    READYnotifications/initialized

    The client confirms that normal operation can begin. Unsupported features stay off the board.

  3. 03
    OPERATElist · read · call

    The pair exchanges requests, responses, and notifications allowed by the negotiated capabilities.

  4. 04
    CLOSEtransport shutdown

    There is no magic goodbye message. The underlying stdio process or HTTP connection signals termination.

LOCAL LANEstdio

The client launches a subprocess and trades newline-delimited JSON-RPC over standard input and output.

REMOTE LANEStreamable HTTP

The server exposes an HTTP endpoint; POST carries messages and SSE may stream server messages.

04 / SCOPE CHECK

A shared socket is not a universal appliance.

Standardization deletes a large class of bespoke glue. It does not delete product judgment, security work, compatibility testing, or delightfully specific bugs.

MCP STANDARDIZES
  • MESSAGE SHAPE

    JSON-RPC requests, responses, errors, and notifications.

  • LIFECYCLE

    Version agreement, capability negotiation, operation, and shutdown.

  • PRIMITIVES

    Common interfaces for tools, resources, prompts, and negotiated features.

  • DISCOVERY

    Ways to list what a server offers before attempting to use it.

  • TRANSPORTS

    Standard local stdio and remote Streamable HTTP lanes.

MCP DOES NOT STANDARDIZE
  • MODEL BEHAVIOR

    MCP does not decide what the model notices or chooses.

  • PERMISSION POLICY

    The host still defines consent, approval, and sandboxing.

  • SERVER SAFETY

    A valid protocol speaker can still be buggy or malicious.

  • SEMANTIC QUALITY

    Two tools with tidy schemas can still mean very different things.

  • UNIVERSAL SUPPORT

    Hosts and servers implement different optional capabilities and extensions.

05 / WHERE IT EARNS RENT

Use MCP where adapters are breeding.

The sweet spot is a growing matrix: several AI hosts, several systems, and repeated need for discovery plus structured access. One tiny one-off API call may not need a protocol ecosystem in a trench coat.

PRIVATE INTEGRATIONShosts × systemsbespoke adapters multiply
SHARED PROTOCOLeach side implements MCPcompatibility still gets tested
A
resources · tools

Developer workbenches

Expose repository resources, search tools, test runners, issue trackers, and focused prompts to an IDE or coding host.

B
resources · prompts

Knowledge desks

Offer handbooks, schemas, support articles, and curated workflows without baking every data connector into the host.

C
tools · resources

Operational cockpits

Let one host discover focused servers for tickets, deployments, calendars, monitoring, or internal workflows.

06 / TRUST & OPERATIONS

The socket is not the lock.

MCP creates a boundary you can reason about. It does not enforce your policy for you. The host, client, server, authorization layer, and upstream systems each keep distinct responsibilities.

  1. 01
    SUPPLY CHAIN

    Trust the server deliberately.

    Treat server code, tool descriptions, annotations, resources, and returned content as untrusted input until you have verified the publisher and behavior.

  2. 02
    HUMAN GATE

    Approval belongs in the host.

    Before a tool call, show the user which server, which tool, which arguments, and the likely effect. MCP defines the call—not your consent interface.

  3. 03
    BLAST RADIUS

    Give every plug a small fuse.

    Use least privilege, narrow roots and scopes, sandbox local processes, validate inputs, and keep dangerous operations separately gated.

  4. 04
    NO TOKEN TOURISM

    Keep credentials in their lane.

    An HTTP token must be intended for the MCP server. Do not pass it through to an upstream API; the server uses a separate upstream credential. Stdio servers get credentials from their environment.

  5. 05
    BORING = GOOD

    Operate it like a real system.

    Add timeouts, cancellation, audit logs, version checks, origin validation for HTTP, secure session handling, and a plan for a server that hangs or disappears.

07 / THE WIRE RECEIPT

A connection, printed on thermal paper.

SDKs wrap this in nicer APIs, but the durable mechanism is a JSON-RPC exchange. This abridged transcript separates negotiated capability families, discovered items, host approval, and the eventual call.

MCP WIRE SERVICESESSION #009-ISSUES
PROTOCOL2025-11-25
  1. 01OUTinitialize
    {
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {
    "protocolVersion": "2025-11-25",
    "capabilities": {},
    "clientInfo": {
      "name": "ai-workbench",
      "version": "1.0.0"
    }
  }
}

    Client proposes a version and advertises what it can support.

  2. 02INinitialize result
    {
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2025-11-25",
    "capabilities": {
      "tools": { "listChanged": true },
      "resources": {}
    },
    "serverInfo": {
      "name": "issue-dock",
      "version": "2.4.0"
    }
  }
}

    Server confirms the version and advertises capability families.

  3. 03OUTtools/list
    {
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list"
}

    Now the client asks which tools actually exist.

  4. 04HOSTuser approval
    server: issue-dock
tool:   create_issue
args:   { "title": "Export fails on CSV" }

    This gate is host product behavior, not a JSON-RPC method.

  5. 05OUTtools/call
    {
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "create_issue",
    "arguments": {
      "title": "Export fails on CSV"
    }
  }
}

    Only after approval does the client send the structured call.

CAPABILITY ADVERTISEDITEM DISCOVEREDUSER APPROVEDCALL SENT ✓
08 / MYTH DETECTOR

Six things the acronym is not wearing.

MCP is useful enough without promoting it to model, agent, security system, integration fairy, and universal remote.

01MYTH

MCP is an agent framework.

No. It defines a protocol for exchanging context and capabilities. The host decides how—or whether—to run an agent loop.

02MYTH

The model connects directly to every server.

The host creates MCP clients and mediates connections. Servers should not receive the whole conversation or inspect sibling servers.

03MYTH

A valid MCP server is a safe MCP server.

Protocol compliance proves message compatibility, not benevolence, correctness, least privilege, or good taste.

04MYTH

Discovery means the model gets everything.

Listing a capability or resource only reveals that it exists. The host still decides what is exposed, read, approved, or placed in context.

05MYTH

One MCP implementation works everywhere.

Versions, optional capabilities, transports, extensions, authorization, and host UX vary. Negotiate, inspect, and test.

06MYTH

MCP replaces API design.

It standardizes the outer conversation. Clear names, useful schemas, idempotency, errors, permissions, and domain semantics remain your craft.

KEEP THIS

MCP is a negotiated protocol boundary. The host keeps the judgment; clients keep connections isolated; servers expose focused capabilities; permissions and credentials keep their own lanes.

Patch another server