AI BENCHNOTES
RAGCHAPTER 005Tool callsAgent loops
AI BENCHNOTES
PLAIN-LANGUAGE VIEWChoose a question.
12 CHAPTERS / EVERYDAY EXAMPLES / NO CODE REQUIRED

Start with a familiar question. Each chapter introduces the standard technical term.

00Plain-language topic guide
CHAPTER 005FUNCTION CALLING

The model requests.Your app decides.

Tool calls: the AI asks.The app checks and executes.

A custom tool call is structured model output containing a function name, arguments, and an identifier. The application validates, authorizes, executes, and returns the result.An AI can ask to look up an order, check a calendar, or take another action. The surrounding app checks the request, decides whether it is allowed, and runs the real tool.

  • PROPOSEDSUGGESTEDby the model
  • CHECKEDby your applicationbefore anything happens
  • RETURNEDREPORTEDas fresh contextso the AI sees the result
01 / OVERVIEW

A call is a proposal, not permission.A request is not permission.

Tool calling gives a probabilistic model a typed interface to deterministic software. The application owns validation, authorization, execution, and error handling.The AI suggests a tool and fills in the details. Ordinary software checks the request, asks for approval when needed, and does the real work.

  1. 01
    PROPOSAL

    The model proposes.asks.

    Selects a declared tool and produces structured arguments.Chooses from the tools it has been shown and fills in the request details.

  2. 02
    CONTROL

    Your application decides.checks.

    Parses, validates, authorizes, confirms, and routes the request.Checks the request, asks for approval when needed, and decides whether it may run.

  3. 03
    EXECUTION

    The tool does.does.

    Runs ordinary code against an API, database, service, or device.Ordinary software carries out the approved action in another service or device.

  4. 04
    RESULT

    The result informs.reports.

    Returns to the conversation so the model can continue from evidence.The outcome comes back, so the AI can respond using what actually happened.

The model proposes a call. The application decides whether and how it runs.

Scope note:hosted or built-in tools may run inside a provider's infrastructure. This chapter focuses on custom, client-side tools, where your application owns the executor and its safety boundary.Scope: some AI services include their own tools. Here we focus on tools your app controls, because your app also controls the safety checks.

02 / INTERACTIVE EXAMPLE

The same application checks every proposed call.

Choose what arrives at the application boundary. The model's proposal changes; the application applies the same gates every time.

TOOL CALLWhere is order 1842?
EXECUTED
MODEL PROPOSAL

toolget_order_status

call_idcall_7f3

arguments
order_id
"ord_1842"
GENERATED TEXT, EVEN WHEN IT LOOKS LIKE CODE
  1. 01
    Schemarequired string
  2. 02
    Authorizesupport:read
  3. 03
    Confirmread-only
  4. 04
    Executeorders service
EXECUTION SIDEORDERS APICONNECTED
TOOL RESULT · CALL_7F3{"status":"out_for_delivery","eta":"today"}
NEXT MODEL TURN

It’s out for delivery and due today.

The schema can catch a wrong type. Only application policy can decide whether this user may do this thing, to this object, right now.

03 / REQUEST AND RESULT SEQUENCE

Tool calling follows a request, execution, and result sequence.The AI requests a tool, the app runs it, and the result comes back.

APIs use different envelope names, but client-side tools share a durable rhythm: definitions in, proposal out, execution elsewhere, result back in.Different AI services use different labels, but the rhythm is the same: show the tools, receive a request, check and run it elsewhere, then return the result.

  1. 01
    APP → MODELAPP → AI

    Declare the callable surfaceShow the available tools

    Send the user’s message plus tool names, descriptions, and parameter schemas.Send the person’s request plus a short description of each tool and the details it needs.

  2. 02
    MODEL → APPAI → APP

    Receive a structured proposalReceive a suggested request

    The model may answer normally—or return one or more calls with names, arguments, and identifiers.The AI may answer normally or ask to use one or more tools, with the details filled in.

  3. 03
    APPAPP

    Cross the control boundaryCheck the request

    Parse, validate, authorize, ask for confirmation when needed, then invoke known code.Check the details, permission, and need for human approval before running known software.

  4. 04
    TOOL → APPTOOL → APP

    Capture the actual resultSave what actually happened

    Success, empty data, timeouts, and failures are all results. Normalize them deliberately.Success, no data, a timeout, and a failure are all real results worth returning clearly.

  5. 05
    APP → MODELAPP → AI

    Return evidence with its call IDReturn the result to the right request

    Add the tool output to the next model input, paired with the call that requested it.Give the AI the tool result and keep it paired with the request that produced it.

  6. 06
    MODELAI

    Continue, don’t assume finishedChoose the next step

    The model can answer, ask a question, repair a bad call, or request another tool.The AI may answer, ask a question, correct a bad request, or ask for another tool.

ABOUT PARALLEL CALLSTWO REQUESTS AT ONCE

Independent calls can run concurrently when IDs keep results paired.Two tools can work at once. Keep each result with the right request.

Independent work—say, weather for two cities—can run concurrently. Keep every call ID attached to its own result, preserve a deterministic return order, and avoid parallelizing actions that depend on one another.Independent work, such as checking two cities’ weather, can happen at the same time. Label each request and result so they cannot be mixed up, and keep dependent steps in order.

04 / TOOL DESIGN

A schema tells the model what it may ask for.

Good definitions reduce guessing. They do not replace runtime validation, authorization, or judgment about side effects.

TOOL DECLARATIONJSON-SCHEMA-LIKE
{
  name: "get_order_status",
  description:
    "Read delivery status for an order
     the signed-in user may view.",
  parameters: {
    type: "object",
    properties: {
      order_id: {
        type: "string",
        pattern: "^ord_[0-9]+$"
      }
    },
    required: ["order_id"],
    additionalProperties: false
  }
}
  1. 01

    Name it like code

    Use a specific verb such as get_order_status, not do_customer_thing.

  2. 02

    Describe when, not just what

    Tool descriptions help the model choose. Include boundaries and important distinctions between similar tools.

  3. 03

    Constrain the arguments

    Required fields, enums, formats, and closed objects turn fuzzy intent into a smaller, testable surface.

TOOL CHOICE

The model may choose. The application sets the menu.

  • Automaticanswer or call a useful tool
  • Requiredproduce some tool call
  • Constrainedchoose only a named subset

Exact controls and names vary by provider. Whichever mode you choose, expose the smallest useful set of tools for this turn.

05 / WHEN TO USE A TOOL

Use tools for external data, exact computation, and real actions.

The model is good at translating messy intent into a candidate action. Let trusted systems own facts, exact computation, and effects.

01

Fresh or private data

Look up orders, inventory, account state, or documents that are not in the model’s input.

get_order_status(…)
02

Deterministic work

Hand exact arithmetic, date logic, parsing, or domain rules to software built for exactness.

calculate_tax(…)
03

Real actions

Create tickets, send messages, schedule meetings, or issue refunds—with explicit gates.

create_support_ticket(…)
04

Specialized capabilities

Search, render a chart, query a database, run code, or operate a domain-specific service.

search_catalog(…)
WHEN A TOOL IS UNNECESSARY

Some tasks need only a model response.

  • The answer is already in the supplied context.
  • The task is purely rewriting, classifying, or formatting.
  • A high-risk action has no meaningful authorization or confirmation path.
  • The available tools overlap so much that the intended action is ambiguous.
06 / SAFETY RULES

Treat every proposed call as untrusted input.

The model can produce useful structured arguments, but the application must independently validate their meaning, authority, and consequences.

  1. 01
    REGISTRY

    Execute from an allowlist.

    Map known names to known handlers. Never eval a model-generated name, import path, URL, SQL fragment, or shell command.

  2. 02
    VALIDATE

    Validate meaning, not only shape.

    Check types and enums, then business invariants: object exists, amount is positive, dates are sensible, fields agree.

  3. 03
    AUTHZ

    Authorize against the real principal.

    Use the signed-in user, tenant, resource, and current policy. The model is neither the user nor an authority.

  4. 04
    CONFIRM

    Put friction before consequences.

    Separate reads from writes. Preview or confirm destructive, costly, external, or difficult-to-reverse actions.

  5. 05
    RELIABILITY

    Design for retries and duplicates.

    Use timeouts, idempotency keys, bounded retries, and explicit success states. A network error may hide a completed action.

  6. 06
    ERRORS

    Return useful errors, not secrets.

    Give the model structured, actionable failure context while redacting tokens, stack traces, private rows, and internal policy details.

SIDE-EFFECT LEDGERMore consequence → more application control
READLookup a delivery ETAauthorize + log
WRITECreate a reversible draftvalidate + authorize + preview
IRREVERSIBLESend, delete, charge, publishconfirm + idempotency + audit
07 / IMPLEMENTATION

The application owns the execution path.

SDKs may automate pieces of this loop. Keep the ownership model visible anyway: the client decides which handler runs and what context returns.

safe-tool-call.tsPROVIDER-NEUTRAL PSEUDOCODE
// Provider-neutral sketch. Adapt the message envelopes to your SDK.
const response = await model.generate({
  messages,
  tools: publicToolDefinitions
});

for (const call of response.toolCalls ?? []) {
  // 1. Resolve only a server-owned handler.
  const tool = toolRegistry.get(call.name);
  if (!tool) throw toolError(call.id, "unknown_tool");

  // 2. Model output is untrusted input.
  const args = tool.schema.parse(call.arguments);

  // 3. The current user and resource decide permission.
  await tool.authorize({user, args});

  // 4. Put a human gate before consequential effects.
  if (tool.requiresConfirmation(args)) {
    await confirmations.require({user, call, args});
  }

  // 5. Execute ordinary application code.
  const result = await tool.execute(args, {
    idempotencyKey: call.id,
    timeoutMs: 8_000
  });

  // 6. Pair evidence with the call that requested it.
  messages.push(response.message, {
    role: "tool",
    toolCallId: call.id,
    content: JSON.stringify(toModelSafeResult(result))
  });
}

// The model gets a new turn with the tool result in context.
const final = await model.generate({messages, tools: publicToolDefinitions});

The call ID is not decoration. It ties a result to the request that produced it, especially when multiple calls are in flight.

Tool output is new context. The model sees the returned data on a later turn; it is not permanent memory and it should still be treated as potentially untrusted content.

08 / MISCONCEPTIONS

Six common misconceptions.

Most tool-calling bugs begin when a convenient API abstraction gets mistaken for a security or execution model.

01

“The model ran my function.”

For client-side tools, it emitted a structured request. Your runtime selected and executed the handler.

02

“The arguments matched the schema, so they’re safe.”

Shape is one gate. You still need semantic validation, authorization, confirmation, and resource limits.

03

“A tool result is the final answer.”

It is evidence added to the next model turn. The model may summarize it, call again, ask a question, or make a mistake.

04

“An error means the agent is done.”

A structured error can help the model repair arguments or choose another path. Bound the retries.

05

“Parallel calls are always faster.”

Only independent calls belong in parallel. Dependent writes need ordering, and every result must keep its call ID.

06

“More tools make the model more capable.”

A crowded, overlapping menu can make selection worse. Expose the smallest relevant tool set per turn.

Let the model interpret intent. Keep execution deterministic, authorized, and observable.

AI BENCHNOTES

For builders who want clear mental models and implementation detail.For curious people who want to understand how AI products work.

Back to top ↑