Week 4
Build this week's exercise, then run the gate.
make check WEEK=4
| Build | Three action tools, one analysis tool and the agent loop, with every wall. Then three ways on three task types, and your model against a frontier one. |
| Files | app/tools/actions.py, app/tools/analysis.py, app/agents/agent.py |
| Measure | make evidence WEEK=4 on a real model, into evidence/week-4.json |
| Record | decisions/week-4.md from the evidence, ai-use/week-4.yml, reflections/week-4.md |
| Guides | The LangGraph comparison · Glossary |
checks fail 7 of 30 checks pass
Red is the normal state of a week you have not built yet. Each name is a test in the file beside it.
Written last: make evidence WEEK=4, then decisions/week-4.md and ai-use/week-4.yml.
The exercise is in your repository, where you can run it:
code weeks/4/README.md make next
Answers that cite their source and decline when the evidence is thin, your own test set, a judge you checked against labels, and the agent's path measured as well as its answer.
An agent that looks up, analyses and acts: a reply, an escalation or a credit, the risky ones only with approval enforced in code. An MCP server, and a routing table that says where an agent and a bigger model pay.
Week 5 defends it. Once the system can act, the content it reads can attack it.
Agency is a cost you place deliberately: where the model chooses, it must be measured, limited and approved. And so is model size.
Read this once (about 25 minutes). Write the sentence in your own words in
reflections/week-4.md, Q1, before you touch code.
| Way | Who decides the steps | Cost | Predictability | Handles a new request? |
|---|---|---|---|---|
| Plain code | the programmer, in advance | zero | total | no: someone writes more code |
| Workflow | the programmer; the model does one job inside a step | a few calls, fixed | high | only if the steps still fit |
| Agent | the model, as it goes | varies per run | low | yes, sometimes, at a price |
An agent is slower, more expensive and less predictable than a workflow, and on some requests it is the only thing that works, because the next step depends on what the last tool returned. The skill is not preferring one; it is saying, task type by task type and from something you measured, which deserves which. That is your routing table.
An agent is a loop: the model picks a tool, you run it, you show it the result, it picks again. The loop needs walls or it will hurt you:
finish ends the run; everything else is a step.The named patterns are variations on where the deciding happens: reason and act (the loop above), plan then execute (decide the steps first, then run them; pro builds it), human in the loop (the approval stop), evaluator and optimiser, orchestrator and workers. Each costs more and predicts less than the loop it replaces.
flowchart LR
q[request] --> d{model picks a tool}
d --> v{"arguments
valid?"}
v -- no --> e["error text
back to the model"] --> d
v -- yes --> a{"writes or spends,
and approved?"}
a -- "not approved" --> stop["stop: needs_approval
nothing happened"]
a -- "read-only or approved" --> t[run the tool] --> r[result back to the model] --> d
d -- finish --> n{"every number in the answer
came from a tool?"}
n -- no --> e
n -- yes --> done[answer]
w{{walls: steps, cost}} -.- d
A model that can write SQL can drop a table, read another desk's tickets, or invent a column. The safe way to let it analyse is a short allowlist of questions, each with parameters, run by you over exactly the rows it may see. Read-only, parameterised, and the result says what was asked. Then enforce, rather than hope, that the number in the answer is the number the query returned: "9" becomes "about a dozen" more often than you would think.
Small models fail first on multi-step planning and on choosing among many tools. A frontier model can be ten to fifty times the cost per call. So the answer is usually a mix, decided task type by task type: the cheap model where it is right, the frontier model only where the measurement says the cheap one fails. You run your trajectory set once on a frontier model through the programme's key, and your routing table gets a second column: which model each task type deserves.
The Model Context Protocol is how a client (an IDE, a desktop app, another agent) discovers and calls your tools without a client written for each. It is a transport and a schema convention. It is not a permission system. Behind an MCP server, three questions matter more than the protocol:
flowchart LR
C["MCP client"] -- "list tools / call tool" --> S["app/mcp_server.py"]
S --> SC{"token's scope
allows this tool?"}
SC -- no --> D1[denied]
SC -- yes --> RL{"rate limit"}
RL -- over --> D2[denied]
RL -- ok --> T["app.tools.call
validation, then approval"]
T -- "needs approval" --> P[needs approval]
T --> R[result, as text]
| Idea | Where it lives | What you do this week |
|---|---|---|
| Action tools | app/tools/actions.py |
Design draft_reply, escalate, issue_credit; say truthfully which write and spend |
| Analysis | app/tools/analysis.py |
Design query_tickets: an allowlist, read-only |
| The door | app/tools/base.py |
Read it: validation, then approval |
| The loop | app/agents/agent.py |
Build it: walls, recovery, the approval stop, the number check |
| Three ways | app/agents/plain.py, workflow.py, scripts/compare_week4.py |
Measure all three on three task types |
| Model size | scripts/model_compare.py |
Your model against a frontier one |
| MCP | app/mcp_server.py, app/mcp_scopes.py |
Serve the tools; try to break your own scopes |
explore/w4_02 showed you.reader and a supervisor token can and cannot do through your MCP server, and how you
proved it.Checked September 2026.
agent.py is this paper's idea.Answers are kept in this browser only, under hub.quiz in local storage. They are never sent anywhere: not to the repository, not to the progress export, not to your mentor, and they have no effect on your route. Clearing your browser data clears them. The explanations are the point, not the score.