Structured Output and Tool Calling · Tool Calling · lesson 5 of 6

The call, execute, return round trip

about 20 minutes · free · runs in your browser

Step 1 of 2

Four steps, and the third is yours

A tool call is not one request. It is a loop with your code in the middle:

You send the question and the tools.
The model answers with finish_reason: "tool_calls" and a name plus JSON arguments. message["content"] is None — there is no prose, which is the first thing that surprises people.
You run the function. The model cannot; it only asked.
You send the result back and get the answer in words.

call = response["choices"][0]["message"]["tool_calls"][0]
name = call["function"]["name"]
args = json.loads(call["function"]["arguments"])   # arguments arrive as a JSON *string*

That last detail catches everyone: arguments is a string containing JSON, not a dictionary, because the model produced text.

Your turn: write run_tool(response, registry) that reads the first tool call out of a response and runs the matching function from registry with the decoded arguments.

You start from this, and edit it in the browser:

import json

import fake_llm

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Look up the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]


def _weather_response():
    """A real tool-calling response, for the tests to hand you."""
    return fake_llm.chat(
        [{"role": "user", "content": "what is the weather in Oslo"}], tools=TOOLS
    )


def run_tool(response, registry):
    """Execute the response's first tool call using registry[name]."""
    return None

Step 2 of 2

Handing the result back

The model asked a question; the tool result is the answer to it. Send it back as a tool message and the model writes the sentence a person actually reads.

messages.append(response["choices"][0]["message"])          # what it asked for
messages.append({"role": "tool", "content": str(result)})   # what you found
final = fake_llm.chat(messages)                             # no tools this time

Note the last call passes no tools. The model already has what it needed; offering the tools again invites it to ask a second time, and a loop with no stopping condition is how a tool-calling program spends forty dollars answering "what is the weather".

Your turn: write answer_with_tool(question, tools, registry) that runs the whole round trip and returns the final text. If the model answers in prose without calling anything, return that prose — a tool that does not fit is not an error.

You start from this, and edit it in the browser: