ImageFirm The FirmStart an enquiry

LibraryAgent engineeringThis sheet

Sheet N°08 · Agent engineering · Anatomy of a run

What happens inside an AI agent

Every agent run is the same nine-stage circuit: understand, plan, retrieve, reason, act, observe, loop, verify, deliver. The stages are cheap to draw and expensive to get right. This sheet shows each one at production depth — what it does, what it needs, where it breaks, and where a human must stay in the chain.

Stages 9 Cross-cutting controls 4 Human gates 3 Failure modes 12 Revision 2026.09

The circuit

One trigger in, one delivered outcome out. The loop in the middle is the agent; everything outside it is scaffolding.

Nine-stage agent run map A trigger enters at the top. Understand and plan run once. Retrieve, reason and act form the inner loop with observe feeding back. Verify sits below the loop as a gate with a human-authority branch, and deliver closes the run. A trace runs down the whole circuit. trace · cost · policy · authority Trigger user · event · schedule · agent 01Understand intent · constraints · context · ambiguity check 02Plan decompose · pick tools · success rule · budget 07 · THE LOOP IS THE AGENT stops on: done · budget · stall · escalate 03Retrieve memory · RAG · task state context budget 04Reason think · choose action or: answer is sufficient 05Act search · API · code · DB MCP · browser · handoff 06Observe parse result · classify error · new info update state · update cost ledger next action refresh context stopping rule fired 08Verify evals · guardrails · confidence · policy HUMAN authority below threshold · irreversible · out of policy fail → replan (bounded) 09Deliver respond · write · trigger workflow · trace attached
01–09

The stages at production depth

01

Understand

What is actually being asked, by whom, under which limits — and is it clear enough to spend money on?

Most agent failures are decided here and discovered at stage 8. The job is to convert a request into a specification the run can be judged against: the intent behind the wording, the constraints that bound it, the context that shapes it, and an explicit call on whether the request is clear enough to proceed.

  • IntentThe outcome behind the words. "Summarise this contract" from legal wants risk; from sales wants terms.
  • ConstraintsHard limits: budget, deadline, jurisdiction, data boundary, tone, format, what must not be touched.
  • ContextWho is asking, their role and permissions, the conversation so far, the system's own remit.
  • Ambiguity checkAn explicit decision: proceed, proceed with stated assumptions, or ask one precise question.
Production reality

What good looks like

  • Intent, constraints and assumptions are written back into the trace as a structured object, not left implicit in the prompt.
  • Identity and permission scope resolve here — the run never discovers later that the requester could not see the data it used.
  • Clarifying questions are rationed: one, precise, only when the cost of guessing exceeds the cost of asking.
  • Untrusted content in the request (pasted files, URLs, forwarded mail) is tagged as data, not instruction.

Where it breaks

  • Literal reading: the agent optimises the sentence, not the outcome.
  • Silent assumptions: a guess made here is presented as a fact at delivery.
  • Instruction smuggling: text inside a document is obeyed as if the user typed it.
  • Over-asking: five clarifying questions for a task the user considered obvious.
02

Plan

How will the goal be reached, with which tools, and how will the run know it is finished?

A plan is a hypothesis about the route, not a contract. Its real deliverable is the stopping rule: the definition of done, the budget the run may spend reaching it, and the conditions under which it should give up or hand over. A plan without a stopping rule is an open-ended bill.

  • DecomposeBreak the goal into steps with explicit dependencies. Parallelise what is independent; sequence what is not.
  • Select toolsPick the smallest tool set that can complete the job. Every tool exposed is attack surface and decision noise.
  • Success criteriaTestable, written down before execution. "Report includes Q3 figures reconciled to source" beats "a good report".
  • Budget & stop ruleMax iterations, token and cost ceiling, wall-clock limit, and what to do when any of them is reached.
Production reality

What good looks like

  • Plans are cheap and revisable — re-planning after new information is normal, but bounded (a maximum number of replans per run).
  • Irreversible steps (send, pay, delete, publish) are flagged at planning time and scheduled behind a checkpoint.
  • Tool selection respects least privilege: read-only tools for research phases, write tools only when the plan reaches the write step.
  • The plan is stored as state, so a crash or restart resumes from the last completed step rather than from zero.

Where it breaks

  • Over-planning: ten steps written, three needed, the rest executed anyway.
  • Tool sprawl: forty tools in scope, the model picks the wrong one for a routine step.
  • No stop rule: the run iterates until the rate limit, not until the goal.
  • Unflagged irreversibles: the destructive step sits in the middle of an autonomous sequence.
03

Retrieve

What does the run need to know that the model does not already know — and how much of it fits in the window?

Retrieval assembles the working context: what the system remembers, what it can look up, and where the task currently stands. The discipline is context budgeting — the window is finite and attention degrades as it fills, so every token loaded should earn its place.

  • MemoryFour kinds, not one: working (this run), episodic (past runs), semantic (facts and preferences), procedural (how-to and skills).
  • Retrieval (RAG)Chunk → embed → search (hybrid: vector + keyword) → rerank → cite. Freshness and access control are part of the pipeline, not afterthoughts.
  • Task stateThe run's own ledger: completed steps, open questions, tool outputs so far, budget consumed, current hypothesis.
Production reality

What good looks like

  • Every retrieved passage carries provenance (source, timestamp, permission scope) and the model is instructed to cite it.
  • Retrieval is filtered by the requester's permissions before ranking, never after.
  • Memory writes are governed: what is remembered, for how long, and who can inspect or delete it — stated, not implicit.
  • Context is compacted between iterations (summaries of old tool output) so the window stays sharp on long runs.

Where it breaks

  • Context stuffing: everything loaded, the important passage lost in the middle.
  • Stale retrieval: last quarter's policy answered as if current.
  • Permission leak: a document the requester cannot open shapes the answer they receive.
  • Memory drift: a wrong "fact" remembered once and reinforced every run thereafter.
04

Reason

Given everything in the window, what is the single best next move — or is the answer already good enough?

This is one model call with a structured output: a thought, a chosen action with arguments, or a declaration that the goal is met. The critical design choice is the answer-readiness check — an explicit path for the model to stop acting, which is the only thing that separates an agent from a loop that runs until it is told to stop.

  • ThinkReasoning before acting: decompose the immediate step, weigh options, estimate what each tool call would reveal.
  • Choose actionExactly one tool and its arguments, validated against a schema before execution. Ambiguous choices trigger a question, not a guess.
  • Answer ready?A deliberate check: do I have what the success criteria require? If yes, exit to verify. If no, name what is missing.
Production reality

What good looks like

  • Model choice is per stage, not per system: a smaller, faster model for routing and tool selection; a frontier model for synthesis and judgement.
  • Structured outputs (typed tool calls, JSON schemas) replace free-text parsing; malformed calls are rejected and retried once with the error shown.
  • Uncertainty is surfaced as a value ("confidence: low — source disagrees with memory"), which stage 8 can act on.
  • Reasoning traces are kept for audit but never shown to end users as if they were the answer.

Where it breaks

  • Action bias: the model calls a tool because tools are available, not because the step needs one.
  • Confident fabrication: an argument invented to satisfy a schema.
  • No exit path: the prompt never offers "done", so the run never says it.
  • Reasoning leak: chain-of-thought exposed as the deliverable.
05

Act

Execute one thing, in a sandbox sized to the step, and prove afterwards that it happened.

Action is where the agent touches the world, so it carries the strictest engineering: least privilege, timeouts, idempotency and a record. Every call is scoped to the minimum permission the step needs, bounded in time, safe to retry, and logged with its inputs and outputs before the result returns to the loop.

  • Search & fetchWeb, internal search, document fetch. Results are untrusted input — sanitised and tagged before re-entering the context.
  • API & MCP toolsTyped interfaces with schemas, auth scoped per run, rate limits respected, errors mapped to a shared taxonomy.
  • Code executionRuns in an isolated sandbox with no network by default, a CPU/time cap, and a clean filesystem per run.
  • Data & systemsDB reads through views, writes through reviewed endpoints; browser automation and human handoff as explicit tools.
Production reality

What good looks like

  • Tools are classified by blast radius (read / write / irreversible) and the classification decides the approval path.
  • Irreversible tools run behind a dry-run first: show the diff, the recipient list, the payment amount — then execute on approval.
  • Credentials are injected by the harness per call and never appear in the model's context.
  • Every call has a correlation ID linking it to the run, the step and the requester.

Where it breaks

  • Credential exposure: an API key visible in context ends up in a log, a summary or an email.
  • Retry storms: a non-idempotent call retried three times sends three invoices.
  • Unbounded execution: generated code loops, downloads, or reaches a system it should not see.
  • Silent side effects: the action succeeded, nobody recorded what it changed.
06

Observe

What came back, what does it mean for the plan, and what did it cost?

Observation turns a raw tool result into something the next reasoning step can use. The under-built part in most agents is error classification: a timeout, a permission denial and a malformed response call for three different next moves, and a loop that treats them all as "try again" will burn its budget on the wrong one.

  • Parse resultValidate against the expected shape; extract the fields the plan needs; summarise or truncate the rest.
  • Classify errorsTransient (retry with backoff), permanent (change approach), permission (escalate), semantic (result valid but wrong).
  • Detect new informationDoes this change the plan, contradict memory, or satisfy a success criterion? Flag deltas explicitly.
  • Update ledgersTask state, cost consumed, iterations used, progress score. The loop reads these before deciding to continue.
Production reality

What good looks like

  • Large tool outputs are stored out of band and referenced by handle; only a summary re-enters the context.
  • Progress is measured against the success criteria from stage 2, so "did that step help?" has a numeric answer.
  • Contradictions between a fresh result and memory are logged as events, not silently resolved either way.
  • Observation records are the primary evidence for post-run audit and evals.

Where it breaks

  • Blind retry: the same failing call repeated until the iteration cap.
  • Output flooding: a 40,000-token API response dumped into the window whole.
  • Success mis-read: a 200 response with an error body treated as done.
  • No progress signal: the run cannot tell it has stopped advancing.
07

Loop

Plan, reason, act, observe — again. The loop is the agent; the controls on the loop are the product.

Everything from stage 2 to stage 6 repeats until a stopping rule fires. What distinguishes a system you can leave running from a demo is not the loop itself but its termination discipline: four independent reasons to stop, each checked every iteration, each with a defined outcome.

  • DoneThe answer-readiness check passed and success criteria are met. Exit to verify.
  • BudgetIterations, tokens, cost or wall-clock exceeded. Exit with partial result clearly marked partial.
  • StallNo measurable progress over N iterations, or the same action repeated. Replan once; if still stalled, escalate.
  • EscalateAn irreversible step, a permission boundary, or confidence below threshold. Hand to a human with full context.
Production reality

What good looks like

  • Iteration caps are set per task class (a lookup gets 3, a research synthesis gets 25), not one global number.
  • Each iteration is checkpointed; a crash resumes, it does not restart.
  • Sub-agents follow the same loop with their own budgets, and their budgets roll up to the parent's ceiling.
  • Loop metrics (iterations per run, stall rate, escalation rate, cost per successful outcome) are dashboarded and reviewed weekly.

Where it breaks

  • Runaway: no cap, or a cap so high it never fires in practice.
  • Ping-pong: two sub-agents handing the same task back and forth.
  • Partial passed as complete: budget exhaustion delivered as success.
  • Escalation without context: the human receives "needs review" and nothing else.
08

Verify

Before anything leaves the system: is it correct, is it allowed, is it confident enough — and whose call is it?

Verification is a gate, not a formality. It runs three independent checks and then makes the one decision the agent must never make for itself: whether this outcome requires a human's authority. In a well-built system, the gate is separate from the loop — a different prompt, often a different model, with no incentive to approve its own work.

  • EvalsAutomated checks against the success criteria: groundedness (claims trace to sources), completeness, format, factual consistency with retrieved data.
  • GuardrailsPolicy checks on the output and the intended action: data classification, PII, jurisdiction, brand and tone, prohibited operations.
  • ConfidenceA calibrated score combining model uncertainty, source agreement and eval results; routed against a threshold set per task class.
Human authority gateRoute to a person when the action is irreversible, when confidence is below the class threshold, when policy is ambiguous, or when the outcome affects someone's rights, money or safety. The handoff includes the trace, the evidence, the proposed action and a one-line recommendation. The person decides; the agent records.
Production reality

What good looks like

  • Verification failures route back to plan with the failure reason attached, and the replan count is bounded.
  • Thresholds are tuned from measured outcomes — approval rates, override rates, incidents — not set once and forgotten.
  • Human reviewers see a decision-ready package, and their decisions feed back into evals as labelled data.
  • Guardrails run on both the deliverable and the trace, so an unsafe intermediate step is caught even if the final text is clean.

Where it breaks

  • Self-grading: the same model, same prompt, asked "is this good?"
  • Rubber-stamp gate: every run escalated, reviewers stop reading.
  • Threshold theatre: a confidence number that is never calibrated against reality.
  • Output-only checks: the answer passes, the deleted rows on the way there do not.
09

Deliver

Produce the outcome, in the form the requester needs, with the evidence attached and the run closed cleanly.

Delivery has three shapes and one obligation. The shapes: respond, write, trigger. The obligation: transparency — the recipient can see what was done, what it rests on, what was assumed, and where a person intervened. A deliverable without its trace is an assertion; with it, it is evidence.

  • RespondAn answer in the conversation, with sources, stated assumptions and any partial-completion flag.
  • WriteA file, record or document — versioned, attributed to the run, reviewable before it replaces anything.
  • TriggerA downstream workflow, ticket, message or transaction — with the same blast-radius rules as stage 5.
Production reality

What good looks like

  • Every deliverable carries a run ID; the full trace is retrievable by that ID for as long as the retention policy requires.
  • Feedback is captured at the point of delivery (accepted, edited, rejected, why) and becomes eval data.
  • Memory writes at close are selective and governed — the run stores what future runs need, not everything it saw.
  • Cost, latency and outcome are emitted as one telemetry event per run, joined to the business result where one exists.

Where it breaks

  • Naked output: a confident answer, no sources, no assumptions, no trace.
  • Overwrite: the write step replaces a human's document with no version to return to.
  • Fire-and-forget triggers: a workflow kicked off with no record of who authorised it.
  • Feedback loss: the user's correction goes nowhere, the same error ships next week.
+

Four controls that run through every stage

The infographic version stops at nine boxes. A production system adds four layers that touch all of them.

Control 1

Observability

If you cannot replay a run, you cannot debug it, audit it or improve it.

  • One trace per run, one span per stage and per tool call
  • Prompts, inputs, outputs and model versions captured
  • Latency and token cost per span
  • Sampled runs reviewed by humans every week
Control 2

Cost governance

Agents spend money in loops. Budgets are set per run, per task class and per tenant.

  • Hard ceilings at run and daily level
  • Model routing: cheapest model that meets the eval bar
  • Cost per successful outcome as the headline metric
  • Alerts on cost anomalies, not just totals
Control 3

Security

An agent is a new principal in your systems. Treat it like a contractor with a badge, not a trusted employee.

  • Least privilege per run, credentials outside the context
  • Untrusted content tagged as data — prompt-injection defence at retrieve and act
  • Sandboxed execution, egress allow-lists
  • Tool blast-radius classification drives approvals
Control 4

Human authority

Three places a person stays in the chain, by design, not by exception.

  • Stage 1: the agent may ask, never assume, on high-stakes ambiguity
  • Stage 7: escalation is a normal exit, with full context
  • Stage 8: irreversible, low-confidence or rights-affecting outcomes are decided by a person
  • The decision and its reason are recorded with the run
12

Where runs actually fail

Each row is a pattern we have seen in real systems, the stage it originates in, and the control that catches it.

FailureOriginWhat it looks likeThe fix
Literal reading01UnderstandThe request is answered word for word; the goal behind it is missed.Write intent back as a structured object; judge the run against it.
Instruction smuggling01 · 03Understand, RetrieveText inside a document or web page is obeyed as if the user typed it.Tag all fetched content as data; never merge it into the instruction channel.
Open-ended bill02PlanNo definition of done; the run stops when the rate limit does.Stopping rule before execution: done, budget, stall, escalate.
Tool sprawl02 · 04Plan, ReasonDozens of tools in scope; the wrong one chosen for a routine step.Expose the minimal tool set per task class; route tool choice through a small model.
Context stuffing03RetrieveEverything loaded; the decisive passage lost mid-window.Context budget per stage; rerank; compact old tool output.
Permission leak03RetrieveA document the requester cannot open shapes the answer they get.Filter by permission before ranking, using the requester's identity.
Action bias04ReasonTools called because they exist, not because the step needs them.Explicit "answer ready" path; reward stopping in evals.
Retry storm05 · 06Act, ObserveA non-idempotent call retried; three invoices sent.Idempotency keys; error taxonomy decides retry vs. change approach.
Credential exposure05ActAn API key in the context surfaces in a log or summary.Harness injects credentials per call; never in the model's window.
Runaway loop07LoopNo progress, no cap that fires; cost climbs quietly.Per-class iteration caps; stall detection; checkpoint every iteration.
Self-grading08VerifyThe model approves its own output; errors pass through.Independent verifier prompt or model; groundedness evals; calibrated thresholds.
Naked output09DeliverA confident answer with no sources, assumptions or trace.Run ID on every deliverable; trace retrievable; assumptions stated.

Unattended-run readiness

Twelve conditions. All of them true before an agent runs without someone watching.

0 of 12 conditions confirmed

Sheet N°08 · A note on the source

The nine-stage circuit is the industry's shared mental model, and it is a good one. What it leaves out is everything that makes an agent safe to leave running: the stopping rule, the error taxonomy, the independent verifier, and the three places a person keeps final authority. This sheet adds those back. An agent that cannot stop, cannot explain itself, or cannot hand over is not autonomous — it is unsupervised.

— ImageFirm, agent engineering

Work with ImageFirm

Bring us the agent you want to run unattended.

We will map it to this circuit, find the stage that is missing, and build the controls that let a person sign off with confidence.

Start an enquiry