Agents and Tool Use · The Loop · lesson 2 of 7
State between steps
about 18 minutes · free · runs in your browser
Memory is a list, and it grows
An agent's memory is the messages list, and every step appends two entries: what the model asked for and what your code found. Ten steps is twenty extra messages, all of them re-sent on every subsequent call — so a long-running agent's cost grows quadratically with its number of steps.
That is not a footnote. It is the reason a task that looks twice as hard costs four times as much, and why "the agent got stuck in a loop" is usually discovered on the invoice.
Two mitigations, both simple:
- Trim the middle. Keep the first message (the task) and the most recent few (the state of play). The middle is usually tool output already summarised into later steps.
- Summarise the trimmed part if it mattered — one message instead of twenty.
Your turn: write trim(messages, keep_recent=4) keeping the first message and the
last keep_recent, and dropping the middle. The task must survive: an agent that forgets
what it was asked will confidently finish the wrong job.
You start from this, and edit it in the browser:
def trim(messages, keep_recent=4):
"""Keep the first message and the last keep_recent, dropping the middle."""
return messages