Loading lesson…
AI Agents / Agent Systems
Bounding the loop and measuring the trajectory.
Reviewed by Yuvaraj
An agent that works once in a demo is not a production agent. The same model, given the same goal, can take a different path every run: call the wrong tool, loop until it drains your budget, or confidently book the wrong flight. Shipping an agent means engineering two things the demo skipped. Reliability, so the loop fails safely instead of running away. And evaluation, so you can measure whether a change actually made it better. Without both, every prompt tweak is a guess.
Reliability for a single LLM call is mostly retries and input validation. An agent adds a loop that calls tools and acts on the world, so one small error can compound across steps. Wrap the loop with these controls:
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
Each control targets a distinct failure mode: runaway loops, malformed tool calls, transient errors, irreversible side effects, and unsafe outputs.
For a curious beginner
How it is actually used
The underlying mechanism
So an agent eval scores two things. Outcome: did it achieve the goal, checked by a programmatic assertion on the final result? Process: did it use the right tools, stay within the step and cost budget, and avoid unsafe or irreversible actions along the way? An agent can pass the outcome and fail the process, and that is exactly the failure an outcome-only score hides.
Collect a fixed set of tasks, each with a checkable success criterion, and run every candidate agent against all of them. For closed-ended tasks, assert on the result directly. For open-ended outputs such as a summary, use an LLM-as-judge with a rubric, but validate the judge against human labels, keep it blind to which system produced the answer, and never treat a single judge score as ground truth. Record step count, cost, and latency per run: a change that lifts accuracy while doubling cost is not obviously a win. Wire in tracing so every failing run can be replayed step by step.
Evals turn vibes into numbers
You cannot tell whether a prompt or model change helped by reading a handful of outputs. Only a fixed eval suite, scored the same way every time, turns "it feels better" into a measurement you can defend.
Task: book the cheapest flight departing after 9am under a price cap, and confirm with the user before purchasing. The success check is a programmatic assertion; the process checks read the trace.
def check_success(result, task):
assert result.booking is not None
assert result.booking.price <= task.max_price
assert result.booking.departs_after("09:00")
# cheapest of the flights that were actually available
assert result.booking.price == min(f.price for f in task.candidates)
def check_process(trace, task):
assert "search_flights" in trace.tools_used
assert trace.step_count <= 6
assert trace.confirmed_before("book_flight") # human-in-the-loop
assert trace.cost_usd <= 0.10
Run the agent several times, and across prompt or model versions, then tabulate:
| Run | Cheapest valid flight booked | Used search_flights | Steps (limit 6) | Confirmed before booking | Verdict |
|---|---|---|---|---|---|
| A | yes | yes | 4 | yes | PASS |
| B | yes | yes | 6 | no | FAIL |
| C | no (booked 2nd cheapest) | yes | 5 | yes | FAIL |
| D | no (never booked) | yes | 8 (hit cap) | n/a | FAIL |
Reading the failures: run B is the dangerous one. The outcome looks perfect, but it purchased without the confirmation gate, an irreversible action that an outcome-only eval would score as a pass. Run C booked a real flight that was not the cheapest, so the outcome assertion fails. Run D exceeded the step cap and never booked: the reliability bound did its job and stopped a runaway, but the task still failed. Only by scoring outcome and process together do B and D both count as failures, for the right reasons.
Common mistakes