AI Engineering

LLM Tool Calling and Agent Loops

The model never runs your code: it asks for a tool call, your app validates and executes it, and a loop with budgets turns that into an agent you can trust.

Intermediate·14 min read·Updated Oct 7, 2026

An LLM (large language model) cannot call an API, query a database or send an email. With tool calling it can only ask: it returns a structured request, a tool name plus JSON arguments, and your application validates the request, runs the tool and sends the result back as another message. An agent is that exchange in a loop: call the model, execute whatever tools it asked for, append the results, and repeat until it answers or a budget runs out. Every guarantee you need, from correct arguments to safe side effects, lives in the code around the model, not in the model.

Context

Language models first acted in the world through text conventions. The ReAct paper (Yao et al., 2022) had a model write lines like Action: search[...], which a harness parsed, executed and pasted back as an Observation. Parsing free text was fragile, so in June 2023 OpenAI added function calling to its API: you describe functions with JSON Schema, and the model returns a dedicated, machine-readable call instead of prose. Every major model API now has an equivalent, usually named tools or tool use. In late 2024 Anthropic published the MCP (Model Context Protocol), an open protocol for exposing tools and data sources to any compatible client, which other providers and IDEs adopted during 2025.

You have met it when a chat assistant searched the web before answering, when a coding assistant ran your tests and read the output, or when a support bot looked up your order. The smallest possible exchange is two messages from the model with your code in between:

one-tool-call (wire format)

// 1. model replies with a request, not an answer
{"tool_call": {"id": "c1", "name": "get_order",
               "arguments": {"order_id": "ord_8F2K"}}}

// 2. your app runs get_order, sends the result back
{"role": "tool", "tool_call_id": "c1",
 "content": {"status": "shipped", "eta": "2026-10-09"}}

// 3. model now answers in plain text
"Your order shipped and should arrive on 9 October."

Tool (function): Something your code can do, described to the model by a name, a description and a JSON Schema for its arguments.
Tool call: The model’s structured request to use a tool: an ID, the tool name and arguments. It is a suggestion, not an action.
Tool result: What your code sends back after running (or refusing) the call, as a message the model reads on its next turn.
Agent loop: Repeatedly calling the model and executing its tool calls until it produces a final answer or a limit is hit.
Workflow: A fixed sequence of model calls and code steps that you designed, as opposed to an agent choosing its own next step.
MCP server: A process that exposes tools, resources and prompts over the Model Context Protocol so any MCP client can use them.

Why it matters

Tools are what turn a model from a text generator into a feature: answering from your data, changing records, running code. They are also where the risk moves. A model that only writes text can be wrong; a model whose requests your code executes can be wrong with side effects, can loop until it has spent a day's budget, or can be steered by text it read in a tool result. Most production failures of AI features are failures of this harness: unvalidated arguments, missing step limits, unbounded context, duplicate writes on retry, and no trace of what the agent actually did.

The loop, step by step

Each turn, the application sends the full conversation plus the list of tool definitions. The model either answers in text, which ends the loop, or returns one or more tool calls. The application looks each one up, validates the arguments, runs it, and appends one result message per call, including failures, before calling the model again. Nothing runs that your code did not decide to run.

The agent loop. The model only produces requests; your app validates and executes them, feeds results back, and checks the budget before every new turn.

agent.ts

type Tool = {
  name: string
  description: string
  schema: Schema                  // zod-like: safeParse(), toJSONSchema()
  run: (args: any, ctx: Ctx) => Promise<unknown>
}

const MAX_STEPS = 8

export async function runAgent(task: string, tools: Tool[], ctx: Ctx) {
  const messages: Message[] = [
    {role: 'system', content: SYSTEM_PROMPT},
    {role: 'user', content: task},
  ]
  for (let step = 0; step < MAX_STEPS; step++) {
    const res = await llm.chat({model: MODEL, messages, tools: specs(tools)})
    messages.push(res.message)
    if (res.toolCalls.length === 0) return res.message.content   // done

    const results = await Promise.all(res.toolCalls.map(async call => {
      const tool = tools.find(t => t.name === call.name)
      if (!tool) return toolResult(call.id, `Unknown tool ${call.name}`)
      const args = tool.schema.safeParse(call.args)
      if (!args.success)
        return toolResult(call.id, `Invalid args: ${args.error.message}`)
      try {
        const out = await tool.run(args.data, ctx)
        return toolResult(call.id, truncate(JSON.stringify(out), 4000))
      } catch (err) {
        return toolResult(call.id, `Tool failed: ${String(err)}`)
      }
    }))
    messages.push(...results)          // one result per call, errors too
  }
  throw new Error(`No answer after ${MAX_STEPS} steps`)
}

Tool definitions are prompts

The model chooses a tool and fills its arguments from three things only: the name, the description and the schema. Those are prompt text, and they deserve the same care. Say when to use the tool and when not to, what each argument means with an example, and what the tool returns. Constrain the schema as tightly as the data allows (enums, formats, required, no extra properties), because every constraint is one less thing the model can get wrong and one more thing you can reject automatically.

refund-tool.json

{
  "name": "refund_order",
  "description": "Refund a delivered order. Call only after the user confirms.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string", "description": "e.g. ord_8F2K"},
      "reason": {"type": "string", "enum": ["damaged", "late", "other"]}
    },
    "required": ["order_id", "reason"],
    "additionalProperties": false
  }
}

Validate twice: shape, then meaning

A schema check proves the arguments are well formed, not that they are allowed. The tool itself must still check business rules with the real user's permissions: does this order belong to the person in the conversation, is it inside the refund window, did they confirm. Treat model output like input from an untrusted client, because text in a document or web page the agent read can push it toward calls the user never asked for.

	Tool calling	Structured output
What it returns	A request to run something, then continues	The final answer in a fixed JSON shape
Turns	Several, with results fed back	One
Use for	Fetching data, taking actions	Extraction, classification, form filling
Your code	Executes, validates, loops	Validates and stores

Side effects, budgets and context

Read-only tools can be retried and run in parallel freely. Tools that change things need the same discipline as any API that may be called twice: a step can time out after the write happened, the loop can be resumed after a crash, and the model can simply ask for the same action again.

refund.ts

export const refundOrder: Tool = {
  name: 'refund_order',
  description: 'Refund a delivered order. Call only after the user confirms.',
  schema: RefundArgs,
  async run({order_id, reason}, ctx) {
    const order = await orders.get(order_id)
    if (order?.customerId !== ctx.userId) return {error: 'not_found'}
    if (!ctx.confirmed('refund', order_id))
      return {status: 'needs_confirmation'}        // model asks the user
    return payments.refund(order_id, {
      reason,
      idempotencyKey: `${ctx.runId}:refund:${order_id}`,  // retry-safe
    })
  },
}

1
Least privilege. Give each agent only the tools its job needs, with credentials scoped to the current user, never an admin token "because it is internal".
2
Confirmation for irreversible actions. Payments, deletions and outbound messages return "needs confirmation" until a human approves in the UI, not in model-generated text.
3
Idempotency keys. Derive them from the run and the action so a repeated call returns the first result instead of acting twice.
4
Sandboxes for code execution. Generated code runs in an isolated environment with no secrets, limited network and a time limit.
5
Budgets. Cap steps, total tokens, wall-clock time and cost per run, and check them before every model call. A loop without limits will eventually find a way to repeat itself.
6
Context hygiene. Every step appends messages, so trim or summarise large tool results and old steps; a single 200 KB API response can crowd out the instructions.

Workflows first, agents when needed

Not every multi-step AI feature should be an autonomous agent. Anthropic's "Building effective agents" (December 2024) separates workflows, where your code fixes the sequence of model calls, from agents, where the model decides the next step. Workflows are cheaper, faster, easier to test and fail in predictable places; agents earn their cost when the number and order of steps genuinely cannot be known in advance.

Pattern	Shape	Good fit
Prompt chaining	Fixed steps, each output feeds the next	Draft, then check, then translate
Routing	Classify, then hand to a specialised path	Support tickets by type
Parallelization	Independent calls at once, then merge	Review a PR file by file
Orchestrator- workers	One call plans subtasks, workers do them	Changes across an unknown set of files
Evaluator- optimizer	Generate, critique, revise in a loop	Writing against clear criteria
Agent	Model picks tools until done	Open-ended tasks with a checkable result

Routeclassify request

Retrievefixed search step

Agent loop≤ 8 steps, scoped tools

Checkrules + evaluator

A typical production shape: deterministic code around one bounded agent loop, rather than the whole request being an agent.

MCP and multi-agent systems

MCP standardises the plumbing: a server exposes tools, resources and prompts once, and any MCP-aware client (a chat app, an IDE, your own agent) can list and call them. It does not change the loop or the trust model; a tool from an MCP server is still a request your client chooses to execute, and a third-party server is code you are trusting. Multi-agent designs, several agents with their own prompts and tools passing work between them, help when subtasks need different tools or context, and cost more tokens, latency and debugging effort. Reach for them after a single agent with good tools has clearly hit its limits.

Pitfalls

Trusting arguments because they match the schema
A well-formed order_id can still belong to someone else. Schema validation catches shape; ownership, limits and permissions must be checked inside the tool with the real user's identity, or the agent becomes a way around your access control.
No step or cost limit
Models can repeat a failing call, alternate between two tools, or keep "checking" forever. Without a hard cap on steps, tokens and time per run, one bad input becomes an outage-sized bill. Stop, return what you have and log the trace.
Throwing on tool errors
An exception on invalid arguments ends the run when the model would have fixed the problem on its next turn. Return errors as tool results with a clear message, and only abort for your own bugs or exhausted budgets.
Unbounded context growth
Each step appends the full tool output. Long runs push the system instructions out of focus, raise cost per turn and eventually overflow the context window. Truncate, paginate or summarise tool results, and keep only what later steps need.
Using an agent where a workflow would do
If the steps are always the same, letting the model choose them adds latency, cost and new failure modes for no benefit. Write the sequence in code and use the model only for the steps that need judgement.

Interview questions

Q1What actually happens when an LLM "calls a function"?

Nothing runs inside the model: it returns a structured message naming a tool and giving JSON arguments, chosen from the definitions you sent. Your application validates the request, executes the tool if it agrees, and sends the result back as a new message. The model then continues with that result in context.

Q2Walk me through implementing an agent loop.

Keep a message list with the system prompt and the task. Each iteration, call the model with the messages and tool definitions; if it returns text, that is the answer. Otherwise, for each tool call, look up the tool, validate the arguments, run it with the user's permissions and append one result per call, errors included. Check step, token, time and cost budgets before every call, trace each step, and stop with a clear error when a limit is reached.

Q3What happens when the model calls a tool with invalid arguments?

The validator rejects them and the loop sends that rejection back as the tool result, with a message specific enough to fix: which field, what was expected. On the next turn the model usually corrects the call. If it keeps failing, the step budget ends the run instead of letting it retry forever.

Q4How do you make tools with side effects safe?

Treat them like a public API called by an untrusted client. Scope credentials to the current user, re-check ownership and business rules inside the tool, require human confirmation in the UI for irreversible actions, and use idempotency keys so a repeated call does not act twice. Code execution goes in a sandbox without secrets.

Q5When would you build a workflow instead of an agent?

Whenever the steps are known in advance. A fixed chain or router is cheaper, faster, easier to evaluate and fails predictably. I use an agent loop only for the part where the number and order of steps depend on what the model finds, and keep it bounded inside otherwise deterministic code.

Q6How do you stop an agent from running forever or overspending?

Hard limits checked before every model call: maximum steps, total tokens, wall-clock time and cost per run, plus detection of the same call repeating. When a limit is hit, stop, return a partial answer or an error to the user, and keep the trace so you can see why it looped.

Q7What is MCP and what problem does it solve?

The Model Context Protocol is an open protocol for exposing tools, resources and prompts from a server to any compatible AI client. It replaces one-off integrations per app with one server many clients can use. It does not make tools trustworthy: the client still decides what to execute, and a third-party server is third-party code.

Key takeaways

The model only requests tool calls; your code validates them, runs them and feeds results back as messages.
An agent is that exchange in a loop, and the loop needs hard limits on steps, tokens, time and cost.
Names, descriptions and schemas are prompts. Constrain schemas tightly and still check business rules inside each tool.
Return tool errors as results so the model can recover; treat side-effecting tools like an API with untrusted callers.
Prefer fixed workflows and use a bounded agent loop only where the steps cannot be known in advance.
MCP standardises how tools are exposed, not whether they are safe to run.

Sources

#Context

#Why it matters

#The loop, step by step

#Tool definitions are prompts

Validate twice: shape, then meaning

#Side effects, budgets and context

#Workflows first, agents when needed