Structured Output and Tool Calling · Tool Calling · lesson 6 of 6
When the tool call is broken
about 18 minutes · free · runs in your browser
Truncated arguments, and tools that do not exist
Two failures land on the same line of code.
The arguments do not parse. A tool call cut off by max_tokens arrives as valid JSON
up to the cut and nothing after it — {"city": "Os. json.loads raises, and
finish_reason is "length" rather than "tool_calls", which is the tell.
The tool does not exist. The model can name a function you never gave it, and
registry[name] raises KeyError — a crash caused by a model's hallucination, in your
process, at whatever hour it happens.
Both are handled the same way: check before you use, and fail with a message that says which of the two it was.
Your turn: write dispatch(response, registry) returning (result, error) — one
of them is always None. Use "bad_arguments" and "unknown_tool" as the error
strings, and "no_tool_call" when the model answered in prose.
You start from this, and edit it in the browser:
import json
import fake_llm
TOOLS = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Look up the current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}
]
def _weather_response():
"""A well-formed tool call."""
fake_llm.reset()
return fake_llm.chat(
[{"role": "user", "content": "what is the weather in Oslo"}], tools=TOOLS
)
def _truncated_response():
"""A tool call cut off mid-argument."""
fake_llm.reset()
fake_llm.break_next("truncated_tool_call")
return fake_llm.chat(
[{"role": "user", "content": "what is the weather in Oslo"}], tools=TOOLS
)
def _prose_response():
"""An answer with no tool call in it at all."""
fake_llm.reset()
return fake_llm.chat([{"role": "user", "content": "What is a token?"}])
def dispatch():
(, )