The Agent Loop, From Scratch
0/7 lessons completed
Overview

Chapter 3: Build one for real · Lesson 6 of 7 · 12 min read

Project: a codebase Q&A agent

Every lesson in one working program: three read-only tools, brakes, and a done-check that verifies every citation.

Time to put every lesson to work. We'll build an agent that answers questions about a codebase, like "Where do we validate coupon codes?", and backs every claim with a path:line citation you can click.

It's a good first project because it's useful on day one and every tool is read-only: the worst it can do is give a wrong answer, which our done-check will catch. No approvals needed.

The question goes into run(question), the loop from lesson 1 with brakes from lesson 3: 20 steps, $0.30 and no repeated calls. The loop calls three read-only tools: list_files, search and read_file. The answer cites src/checkout/coupons.py:42, and a done-check verifies every cited file and line exists.
The whole agent. Every box is something you built in an earlier lesson.

The full program is about 100 lines of plain Python 3.9+, in four parts. As in every lesson, call_model and cost_of stand for your model API.

Part 1: the tools

Three tools, each following lesson 2's rules: small results, a hint when there's more, and errors that explain themselves.

import json
import re
from pathlib import Path

ROOT = Path(".").resolve()
SKIP = {".git", "node_modules", ".venv", "__pycache__"}


def project_path(path):
    p = (ROOT / path).resolve()
    if not p.is_relative_to(ROOT):
        raise ValueError(f"{path} is outside the project")
    return p


def walk(directory):
    for p in sorted(project_path(directory).rglob("*")):
        if p.is_file() and not SKIP & set(p.relative_to(ROOT).parts):
            yield p


def list_files(directory="."):
    files = [str(p.relative_to(ROOT)) for p in walk(directory)]
    more = f"\n…and {len(files) - 50} more. List a subdirectory." if len(files) > 50 else ""
    return "\n".join(files[:50]) + more or "No files."


def search(pattern, directory="."):
    regex, hits = re.compile(pattern), []
    for p in walk(directory):
        for n, line in enumerate(p.read_text(errors="ignore").splitlines(), 1):
            if regex.search(line):
                hits.append(f"{p.relative_to(ROOT)}:{n}: {line.strip()[:120]}")
    if not hits:
        return "No matches. Try a shorter pattern, or list_files to look around."
    more = f"\n…{len(hits) - 30} more. Use a narrower pattern." if len(hits) > 30 else ""
    return "\n".join(hits[:30]) + more


def read_file(path, start=1):
    lines = project_path(path).read_text(errors="ignore").splitlines()
    end = min(start + 79, len(lines))
    shown = "\n".join(f"{n}: {lines[n - 1]}" for n in range(start, end + 1))
    return shown + (f"\n…{len(lines)} lines in total." if end < len(lines) else "")


TOOLS = {"list_files": list_files, "search": search, "read_file": read_file}

A few details worth copying into your own tools:

  • project_path fences the agent in. The model chooses the paths, so it can ask for ../../.ssh/id_rsa. Resolve the path and refuse anything outside the project. The ValueError turns into an ERROR: result, and the model moves on.
  • Every result is capped: 50 files, 30 matches, 80 lines. And each cap tells the model how to narrow down, instead of silently cutting off.
  • "No matches" comes with a next step. Remember the trace in lesson 5, where the agent searched for the same thing three times? That's the fix.
  • read_file numbers its lines, so the model can cite path:line without counting.

Part 2: describing the tools

A tiny helper keeps the schemas short. Read the descriptions as the model will: they say what comes back and what the arguments look like.

def schema(name, description, required=(), **params):
    return {
        "name": name,
        "description": description,
        "parameters": {"type": "object", "properties": params, "required": list(required)},
    }


SCHEMAS = [
    schema("list_files", "List files under a directory of the project (max 50).",
           directory={"type": "string", "description": "Relative path, e.g. 'src'. Default '.'"}),
    schema("search", "Regex search across the project. Returns path:line: text for each match.", ["pattern"],
           pattern={"type": "string", "description": "Python regex, e.g. 'def apply_coupon'"},
           directory={"type": "string", "description": "Limit to this directory. Default '.'"}),
    schema("read_file", "Read 80 numbered lines of a file, starting at `start`.", ["path"],
           path={"type": "string", "description": "Relative path, as returned by search"},
           start={"type": "integer", "description": "First line to show. Default 1"}),
]

SYSTEM = (
    "You answer questions about the code in this project. Use the tools to look; never guess. "
    "Cite every claim as path:line, e.g. src/app.py:42. If you can't find it, say so."
)
MAX_STEPS, BUDGET_USD, MAX_REPEATS = 20, 0.30, 2

Part 3: the loop, with brakes

This is lesson 1's loop with lesson 2's error handling and three of lesson 3's brakes: a step limit, a cost budget and repeat detection.

def run(question, call_model):
    messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}]
    seen, spent = {}, 0.0
    for step in range(MAX_STEPS):
        reply = call_model(messages, tools=SCHEMAS)
        messages.append(reply)
        spent += cost_of(reply.usage)
        if not reply.tool_calls:
            return reply.content
        if spent > BUDGET_USD:
            return f"Stopped: spent ${spent:.2f} without an answer."
        for call in reply.tool_calls:
            key = (call.name, json.dumps(call.arguments, sort_keys=True))
            seen[key] = seen.get(key, 0) + 1
            if seen[key] > MAX_REPEATS:
                result = "ERROR: you already made this exact call. Try something different."
            else:
                try:
                    result = TOOLS[call.name](**call.arguments)
                except Exception as e:
                    result = f"ERROR: {type(e).__name__}: {e}"
            messages.append({"role": "tool", "tool_call_id": call.id, "content": str(result)})
    return "Stopped: step limit reached."

Compare it with the 30 lines from lesson 1. The shape hasn't changed at all. Every addition is a guard around the same loop.

Part 4: checking "done"

The model says it's finished when it stops calling tools. For this agent, "finished" has a definition we can check in code: every citation points at a real file and a real line. If not, we send it back once with the problems listed.

CITATION = re.compile(r"([\w./-]+\.\w+):(\d+)")


def citation_problems(answer):
    cited = CITATION.findall(answer)
    if not cited:
        return ["the answer has no path:line citations"]
    problems = []
    for path, line in cited:
        try:
            lines = project_path(path).read_text(errors="ignore").splitlines()
        except (OSError, ValueError):
            problems.append(f"{path} does not exist")
            continue
        if not 1 <= int(line) <= len(lines):
            problems.append(f"{path} has no line {line}")
    return problems


def ask(question, call_model):
    answer = run(question, call_model)
    if answer.startswith("Stopped:"):
        return answer
    problems = citation_problems(answer)
    if problems:
        answer = run(f"{question}\n\nA first answer had bad citations ({'; '.join(problems)}). "
                     "Check the files and answer again.", call_model)
    return answer

This doesn't prove the answer is right, only that it's grounded in files that exist. That's still a big step up: an answer that cites coupons.py:42 takes you seconds to verify; an answer that cites nothing takes you as long as finding it yourself.

Running it

print(ask("Where do we validate coupon codes?", call_model))

A typical run takes four or five steps: one search for something like coupon, one or two read_file calls to confirm, then the answer. If it's taking fifteen, open the trace (lesson 5). The cause is almost always a tool result that didn't help it decide what to do next.

Make it yours

Some next steps, roughly in order of usefulness:

  1. Add the trace from lesson 5. Call log_step inside run, and you can replay any confusing answer.
  2. Compact old results (lesson 4) if you point it at a large codebase and runs get long.
  3. Add a write_notes tool so it can keep track across a long investigation.
  4. Add a tool that changes something, like open_issue. Now lesson 3's human approval matters: put it in NEEDS_APPROVAL and ask before it runs.

Recap

  • The project is lesson 1's loop plus guards. Nothing new, just everything at once.
  • Fence file access in, cap every result, and make "no results" say what to try next.
  • A done-check that verifies citations turns "trust me" into "check me".

Next, the last lesson: the same loop, inside the frameworks you'll meet in the wild.