Loading lesson…
AI Agents / Agent Anatomy
How a model calls functions and your runtime executes them.
Reviewed by Yuvaraj
A language model on its own can only produce text. It cannot check today's weather, query your database, or send an email. Tool use is the bridge: you hand the model a menu of functions it may call, and when it decides one is needed, it emits a structured request naming the function and its arguments. Your code, not the model, runs that function and feeds the result back. Orchestration is the layer above: how you arrange one or many tools, route between them, and sequence the calls into a reliable flow.
The single most important fact is that the model never executes anything. It emits structured output describing what it wants called; your runtime does the executing and returns an observation the model reads on its next turn.
The loop, precisely:
For a curious beginner
The model is filling out a request form, not pressing the button. It writes down which tool and what to pass; someone else carries the form to the machine and runs it.
How it is actually used
A tool call is just structured output, a JSON object with a name and arguments, that your SDK parses. Your application code owns execution, error handling, timeouts, and permissions. The model's turn ends when it emits the call and resumes only after you supply the result.
The underlying mechanism
Constrained decoding restricts token sampling at each step to tokens valid under the schema's grammar, so the emitted string is guaranteed to parse as JSON matching the parameter types. The model still only assigns probabilities over tokens; it never invokes a function.
The model picks tools entirely from their names, descriptions, and parameter schemas, it never sees your implementation, so the description is effectively a prompt. Say what the tool does, when to use it, and what each parameter means. Constrain the schema with types, enums, and so an invalid call is hard to even express.
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
required{
"name": "get_weather",
"description": "Get the current weather for a location. Use when the user asks about current conditions or temperature. Do not use for multi-day forecasts.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City and country, e.g. 'Paris, France'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit to report"
}
},
"required": ["location"]
}
}
The user asks: "What's the weather in Paris right now?" The model decides get_weather applies and emits:
{
"tool_call": {
"name": "get_weather",
"arguments": { "location": "Paris, France", "unit": "celsius" }
}
}
Your runtime validates the arguments, calls the real weather API, and returns the result as an observation:
{
"role": "tool",
"name": "get_weather",
"content": {
"temperature": 14,
"unit": "celsius",
"conditions": "light rain"
}
}
The model reads that observation and composes the final answer: "It's currently 14°C in Paris with light rain, bring an umbrella."
When a turn's tool calls are independent, weather for three cities, the model can emit several at once and your runtime runs them concurrently, returning all observations together. When one call's arguments depend on another's result, look up a user ID, then fetch that user's orders, the calls must span separate turns, because the model needs the first observation before it can form the second call. Wrap slow tools in timeouts so a single hanging call cannot stall the whole loop.
Start with a single agent and a small toolbox. Reach for a router when tool selection gets unreliable, and for a graph or state machine when a task has real branching, retries, or approval gates.
Tools are where an agent stops being a text generator and touches the real world, spending money, mutating data, sending messages. Scope every tool's permissions to the minimum it needs: a read-only tool should hold read-only credentials. Treat model-supplied arguments as untrusted input and validate them exactly as you would a request from an anonymous user, because in effect that is what they are.
Common mistakes
run_sql or shell tool is a huge blast radius; prefer narrow, specific tools.