Where the agent's state lives

In lesson 1 we said the messages list is the agent's memory. That works for a five-step task. For a fifty-step task it breaks in two ways:

• It gets too long. Every model has a context limit, and well before you hit it, answers get slower, pricier and sloppier. The model pays less attention to things buried in the middle.
• It disappears. When the process crashes or the run ends, the list is gone. The next run starts from zero.

So we split state in two: what the model needs right now, and what the system needs to keep.

Short-term: keep the message list lean

Three techniques, from simplest to most involved.

1. Shrink old tool results. A tool result was useful when it arrived. Ten steps later the model usually only needs to know it happened.

def compact_old_results(messages, keep_last=6):
    for m in messages[:-keep_last]:
        if m["role"] == "tool" and len(m["content"]) > 500:
            m["content"] = m["content"][:200] + " …[trimmed, re-run the tool if needed]"

2. Summarise the middle. When the list passes a size threshold, replace the oldest chunk with a short summary written by the model. Keep the original task and the most recent steps word-for-word.

if count_tokens(messages) > 60_000:
    head, middle, tail = messages[:1], messages[1:-8], messages[-8:]
    summary = call_model([{"role": "user", "content":
        "Summarise these agent steps: decisions made, facts learned, "
        "what is still open. Be concrete.\n\n" + render(middle)}])
    messages = head + [{"role": "user", "content": "Progress so far:\n" + summary.content}] + tail

3. Keep a scratchpad. Give the agent a write_notes tool and a notes file. Ask it to record decisions and open questions as it goes. Notes survive any trimming because they live outside messages.

Long-term: save state outside the loop

Anything you'd be upset to lose in a crash belongs in a file or database, written as you go, not at the end:

def save_checkpoint(run_id, step, messages, notes):
    with open(f"runs/{run_id}.json", "w") as f:
        json.dump({"step": step, "messages": messages, "notes": notes}, f)

On start-up, look for a checkpoint and resume instead of starting over. Combine this with lesson 3's brakes and a crash costs you one step, not the whole run.

Watch out for side effects on resume. If the crash happened right after send_email ran but before the checkpoint was saved, resuming will send it again. Save a checkpoint before irreversible tools, and record their results immediately after.

A simple rule of thumb

Recap

• messages is short-term memory: keep the task and recent steps word-for-word, summarise the rest.
• Anything you'd hate to lose goes outside the loop, written as you go.
• Checkpoint before irreversible tools, so a resume doesn't repeat them.

You now have every piece of a real agent loop: model calls, tools, brakes and state. The last chapter puts them to work: first, how to find out what went wrong when an agent misbehaves.