AI BENCHNOTES
HarnessesCHAPTER 009MCPEvals
AI BENCHNOTES
PLAIN-LANGUAGE VIEWChoose a question.
12 CHAPTERS / EVERYDAY EXAMPLES / NO CODE REQUIRED

Start with a familiar question. Each chapter introduces the standard technical term.

00Plain-language topic guide
CHAPTER 009connection protocol

MCP:a shared protocolfor context and tools.

MCP:a common way for AI appsto connect to tools.

Model Context Protocol gives AI applications a shared way to discover and use external context and tools. It standardizes messages and lifecycle behavior; hosts still own permissions and trust decisions.MCP is a common way for AI apps to connect to outside information and tools. A compatible service still needs appropriate permissions and a trust decision.

Try a connection example Show me the trust boundary
  • 1:1ONE LINKclient to serverfor each connected service
  • JSON-RPCSHARED FORMATmessage layerrequests use the same shape
  • ≠ TRUSTpermission is separateaccess still needs a decision
01 / OVERVIEW

MCP standardizes the conversation, not the decision.MCP gives AI tools one shared plug shape.

A host opens a dedicated client connection to each server. They agree on a protocol version and supported capabilities, then exchange typed requests, responses, and notifications. The host still decides what reaches the model and what is allowed to run.The AI app opens one organized connection to each service. They agree on what the service can offer, but the app still decides what the AI may see and what actions may run.

01
HOSTAI APPOwns the user experience and policyControls the AI app

The AI application the user touches. It manages the model, permissions, context, consent, and a fleet of clients.The app a person uses. It manages the AI, permissions, approvals, and all of the connected services.

One host, many connections.One app, many connections.
02
CLIENTCONNECTIONMaintains one server sessionConnects to one service

A protocol component created by the host. It negotiates and maintains one dedicated session with one server.A dedicated connection between the AI app and one service. It keeps their conversation organized.

One client, one server.One connection, one service.
03
SERVERSERVICEExposes focused capabilitiesOffers information or actions

A local process or remote service exposing focused resources, prompts, tools, and other negotiated features.A local or online service that offers focused information and actions, such as reading files or updating a calendar.

Useful does not mean trusted.Useful still requires permission.
02 / INTERACTIVE EXAMPLE

The protocol stays consistent while server capabilities change.

Connect the host to one server. Initialization and discovery follow the same pattern; capabilities, authorization, and requests differ.

MCP SESSION / ISSUES NEGOTIATED
HOSTAI APPLICATION
USER INTENT

Turn this bug report into a tracked issue.

HOST KEEPSconversationmodelconsent
  1. 01
    initialize

    Agree on version; exchange capability categories.

  2. 02
    discover

    */list reveals the actual inventory.

  3. 03
    host gate

    Mutation ahead. The host shows the server, tool, and arguments to the user before sending.

  4. 04
    tools/call

    create_issue { title, body }

  5. 05
    response

    Issue #842 created · URL returned

REMOTE SERVICEStreamable HTTP
MCP SERVERIssue tracker
ADVERTISED CAPABILITIES
TRP
DISCOVERED INVENTORY
  • TOOLcreate_issue
  • RESOURCEproject://roadmap
  • PROMPTtriage_bug
AUTHORIZATION BOUNDARY

The client’s OAuth token is audience-bound to this MCP server. Any upstream tracker credential is a separate token.

action completedIssue #842 created · URL returned

Same protocol shape. Different capability set, credential path, and operational risk.

03 / CONNECTION MODEL

The host owns every MCP connection.The AI app manages every connection.

“The model connects to MCP” is convenient shorthand, but it erases the useful boundary. The host creates clients, each client maintains one server session, and servers remain isolated from the full conversation and from one another.The AI itself does not plug directly into every service. The app creates one separate connection for each service and decides what information crosses each boundary.

SESSION LOGCONNECTION STEPSNegotiate capabilities before use.Agree before doing any work.

Capability negotiation says which feature families are legal. Discovery methods such as tools/list reveal the actual items inside those families.First they agree on broad abilities. Then the app asks for the exact information and actions this service offers.

  1. 01
    INITMEETinitializesay hello

    The client proposes a protocol version, capabilities, and identity. The server replies with its supported version, capabilities, and identity.The app and service introduce themselves and agree on which shared version and features they both understand.

  2. 02
    READYREADYnotifications/initializedconfirm

    The client confirms that normal operation can begin. Unsupported features stay off the board.The app confirms that work can begin. Features they did not agree on remain unavailable.

  3. 03
    OPERATEWORKlist · read · calllist · read · act

    The pair exchanges requests, responses, and notifications allowed by the negotiated capabilities.The app asks what is available, reads information, or requests an allowed action.

  4. 04
    CLOSECLOSEtransport shutdowndisconnect

    The underlying stdio process or HTTP connection signals termination; MCP does not define a separate shutdown message.When the underlying connection closes, the session is over.

LOCAL LANESAME COMPUTERstdiodirect local link

The client launches a subprocess and trades newline-delimited JSON-RPC over standard input and output.The app starts a local service and exchanges messages with it directly on the same machine.

REMOTE LANEACROSS THE INTERNETStreamable HTTPweb connection

The server exposes an HTTP endpoint; POST carries messages and SSE may stream server messages.The app and service exchange messages over an ordinary web connection.

04 / WHAT MCP STANDARDIZES

MCP standardizes integration mechanics, not product behavior.

Standardization removes a large class of bespoke protocol code. Product judgment, security work, and compatibility testing still remain.

MCP STANDARDIZES
  • MESSAGE SHAPE

    JSON-RPC requests, responses, errors, and notifications.

  • LIFECYCLE

    Version agreement, capability negotiation, operation, and shutdown.

  • PRIMITIVES

    Common interfaces for tools, resources, prompts, and negotiated features.

  • DISCOVERY

    Ways to list what a server offers before attempting to use it.

  • TRANSPORTS

    Standard local stdio and remote Streamable HTTP lanes.

MCP DOES NOT STANDARDIZE
  • MODEL BEHAVIOR

    MCP does not decide what the model notices or chooses.

  • PERMISSION POLICY

    The host still defines consent, approval, and sandboxing.

  • SERVER SAFETY

    A valid protocol speaker can still be buggy or malicious.

  • SEMANTIC QUALITY

    Two tools with tidy schemas can still mean very different things.

  • UNIVERSAL SUPPORT

    Hosts and servers implement different optional capabilities and extensions.

05 / WHEN TO USE MCP

Use MCP when several hosts need structured access to several systems.

MCP is most useful when several AI hosts need discovery and structured access across several systems. A single, stable API call may be simpler as a direct integration.

PRIVATE INTEGRATIONShosts × systemsbespoke adapters multiply
SHARED PROTOCOLeach side implements MCPcompatibility still gets tested
A
resources · tools

Development tools

Expose repository resources, search tools, test runners, issue trackers, and focused prompts to an IDE or coding host.

B
resources · prompts

Knowledge systems

Offer handbooks, schemas, support articles, and curated workflows without baking every data connector into the host.

C
tools · resources

Operational integrations

Let one host discover focused servers for tickets, deployments, calendars, monitoring, or internal workflows.

06 / SECURITY AND OPERATIONS

The host owns consent, permissions, and operational safeguards.

MCP creates a boundary you can reason about. It does not enforce your policy for you. The host, client, server, authorization layer, and upstream systems each keep distinct responsibilities.

  1. 01
    SUPPLY CHAIN

    Trust the server deliberately.

    Treat server code, tool descriptions, annotations, resources, and returned content as untrusted input until you have verified the publisher and behavior.

  2. 02
    HUMAN GATE

    Approval belongs in the host.

    Before a tool call, show the user which server, which tool, which arguments, and the likely effect. MCP defines the call—not your consent interface.

  3. 03
    LEAST PRIVILEGE

    Limit each connection’s authority.

    Use least privilege, narrow roots and scopes, sandbox local processes, validate inputs, and keep dangerous operations separately gated.

  4. 04
    CREDENTIALS

    Separate MCP and upstream credentials.

    An HTTP token must be intended for the MCP server. Do not pass it through to an upstream API; the server uses a separate upstream credential. Stdio servers get credentials from their environment.

  5. 05
    RELIABILITY

    Operate it like a real system.

    Add timeouts, cancellation, audit logs, version checks, origin validation for HTTP, secure session handling, and a plan for a server that hangs or disappears.

07 / PROTOCOL EXAMPLE

A JSON-RPC session from initialization to tool call.

SDKs wrap this in nicer APIs, but the durable mechanism is a JSON-RPC exchange. This abridged transcript separates negotiated capability families, discovered items, host approval, and the eventual call.

EXAMPLE MCP SESSIONSESSION #009-ISSUES
PROTOCOL2025-11-25
  1. 01OUTinitialize
    {
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {
    "protocolVersion": "2025-11-25",
    "capabilities": {},
    "clientInfo": {
      "name": "ai-workbench",
      "version": "1.0.0"
    }
  }
}

    Client proposes a version and advertises what it can support.

  2. 02INinitialize result
    {
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2025-11-25",
    "capabilities": {
      "tools": { "listChanged": true },
      "resources": {}
    },
    "serverInfo": {
      "name": "issue-dock",
      "version": "2.4.0"
    }
  }
}

    Server confirms the version and advertises capability families.

  3. 03OUTtools/list
    {
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list"
}

    Now the client asks which tools actually exist.

  4. 04HOSTuser approval
    server: issue-dock
tool:   create_issue
args:   { "title": "Export fails on CSV" }

    This gate is host product behavior, not a JSON-RPC method.

  5. 05OUTtools/call
    {
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "create_issue",
    "arguments": {
      "title": "Export fails on CSV"
    }
  }
}

    Only after approval does the client send the structured call.

CAPABILITY ADVERTISEDITEM DISCOVEREDUSER APPROVEDCALL SENT ✓
08 / MISCONCEPTIONS

Six common misconceptions about MCP.

MCP has a specific scope: protocol lifecycle, discovery, and structured exchange between hosts and servers.

01MYTH

MCP is an agent framework.

No. It defines a protocol for exchanging context and capabilities. The host decides how—or whether—to run an agent loop.

02MYTH

The model connects directly to every server.

The host creates MCP clients and mediates connections. Servers should not receive the whole conversation or inspect sibling servers.

03MYTH

A valid MCP server is a safe MCP server.

Protocol compliance proves message compatibility, not benevolence, correctness, least privilege, or good taste.

04MYTH

Discovery means the model gets everything.

Listing a capability or resource only reveals that it exists. The host still decides what is exposed, read, approved, or placed in context.

05MYTH

One MCP implementation works everywhere.

Versions, optional capabilities, transports, extensions, authorization, and host UX vary. Negotiate, inspect, and test.

06MYTH

MCP replaces API design.

It standardizes the outer conversation. Clear names, useful schemas, idempotency, errors, permissions, and domain semantics remain your craft.

SUMMARY

MCP is a negotiated protocol boundary. The host keeps the judgment; clients keep connections isolated; servers expose focused capabilities; permissions and credentials keep their own lanes.

Try another server