AI Workflows & Automation / Reliability
Checkpoints where a person must approve.
Reviewed by Yuvaraj
Automation is a bargain: you trade human judgment for speed and scale. Most of the time that bargain is good, but some decisions are too consequential to hand to a model running unattended. Human in the loop is the deliberate practice of inserting a person at exactly those points, so a machine handles the routine flow while a human catches the cases that would be expensive to get wrong. The skill is not "add approvals everywhere." It is knowing which steps earn a checkpoint and building the plumbing so the workflow can wait for an answer without falling over.
Full automation has the lowest latency and the highest throughput: the workflow decides and acts in milliseconds, with no one to wait on. Its failure mode is that a wrong or low-confidence decision executes anyway, and by the time you notice, the money has moved or the email has gone out. A human checkpoint inverts both properties. It adds latency (minutes to hours, bounded by reviewer availability) and caps throughput at what your reviewers can process, but it stops high-stakes mistakes before they happen, while they are still cheap to undo.
Gate an action when one or more of these is true:
Reversible, cheap, internal, high-confidence actions should stay fully automated. Gating those just adds cost and delay.
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
All three depend on pause/resume mechanics: when a run hits a gate it must persist its state (inputs, intermediate results, the proposed action) to durable storage, stop consuming compute, and later resume from that exact checkpoint when the decision arrives. If state lives only in memory, a restart during the wait loses the whole run.
An AI agent proposes refunds. The routing rule:
def route_refund(amount, confidence):
if amount > 200: # always human, any confidence
return "escalate"
if amount < 20 and confidence >= 0.9: # fast path
return "auto_approve"
return "review_queue" # 20-200, or low confidence
Tracing one request of each kind:
| Request | Amount | Confidence | Branch | Outcome |
|---|---|---|---|---|
| A | 12 dollars | 0.96 | auto-approve | Refunded instantly, no human involved |
| B | 85 dollars | 0.88 | review queue | Persisted and paused; resumes when a reviewer approves |
| C | 640 dollars | 0.93 | escalate | Always routed to a human, high confidence notwithstanding |
Request C shows why amount and confidence are separate levers: a confident model is still not allowed to move 640 dollars alone.
Tune the thresholds with data
Start conservative, then move the boundaries using the queue's own history. If reviewers approve nearly every 20-to-50-dollar case, raise the auto-approve ceiling; if escalations keep catching errors, lower it.
Common mistakes