Human-in-the-loop is a product feature

Somewhere along the way, "human-in-the-loop" became something teams say apologetically, as if needing a person means the AI fell short. In production it is closer to the opposite. Systems that know when to ask are the ones customers, auditors, and on-call engineers learn to trust.

The research backs this up. In Measuring Agents in Production, 68% of the deployed agents studied hand control to a person within ten steps. Human involvement is not a temporary crutch in those systems. It is part of the design.

If people are part of the design, the handoff deserves the same craft as the model loop. Most teams give it almost none.

Anatomy of a good approval request

A poor approval request is a notification that says "Agent needs review" and links to a log. The reviewer has to reconstruct what happened, which takes long enough that they either rubber-stamp it or ignore it.

A good approval request is a small, complete decision. Here is an illustrative one for a hypothetical agent that issues account credits:

Proposed action: Issue a $40 credit to account 7731.

Why: The customer reported two failed deliveries this month (tickets 551 and 562). Policy section 4.2 allows a credit of up to $50 for repeated failed deliveries.

Evidence: Delivery log excerpt; policy passage; customer message.

Risk class: Financial, reversible within 30 days.

Options: Approve · Edit amount · Reject with reason.

Each element has a job. The proposed action states exactly what will happen, with the real arguments. The reasoning and evidence let the reviewer verify instead of trust. The risk class tells them how carefully to look. The options make editing as easy as approving, which matters more than people expect: if the only choices are approve and reject, reviewers wave through marginal proposals to avoid redoing the work.

Every decision, including edits, is stored with the reviewer's identity and a timestamp. That record is your audit trail and, later, your evidence for deciding where autonomy can expand.

Tier actions by risk, not by tool

Rather than a blanket rule that people approve everything, classify actions:

The tiers map onto risks named in the OWASP Top 10 for Agentic Applications, particularly tool misuse, identity and privilege abuse, and human-agent trust exploitation, where a persuasive agent talks a person into approving something they should not.

Approval wait is user-facing latency

LangChain's State of Agent Engineering survey ranks latency second among barriers to production, behind quality. In agent systems with approvals, the slowest component is often the queue of people, not the model.

If a customer waits six hours for a credit because the approval sat in someone's inbox, the system is slow no matter how fast the model is. So:

• Set a target time-to-decision for each tier, and measure it.
• Route approvals to people who are actually on shift, with a fallback reviewer.
• Tell the end user what is happening: "A team member is reviewing this; expect an answer within an hour."
• Decide in advance what happens when an approval expires. Escalate or cancel safely; never auto-approve.

Four numbers that show whether it works

• Accept rate: how often proposals are approved unchanged. Very high may signal rubber-stamping; very low means the agent is not ready.
• Edit distance: how much reviewers change the proposals they approve. Shrinking edits are the strongest evidence for expanding autonomy.
• Time to decision: per tier, at the median and at the slow tail.
• Escape rate: problems discovered after approval. Each one becomes a test case and a rubric update.

Reviewers need onboarding too

Approvers are part of the system, so treat them that way. Give new reviewers a short calibration session: a dozen past proposals, some good and some subtly wrong, scored and discussed together. Publish the rubric they are applying. Rotate reviewers so no one person becomes the bottleneck, and watch for fatigue; an approver who accepts eighty proposals an hour has stopped reviewing. If the volume is that high, the tier is probably wrong.

Earning autonomy

Autonomy should expand the way permissions do in a well-run organization: narrowly, on evidence, and reversibly. When a category of proposal shows a high accept rate, small edits, and a low escape rate over a meaningful period, move it down a tier and keep sampling. When something slips through, move it back up without debate. Run this way, the approval step is not a sign the agent is weak. It is how the agent earns trust.

Sources cited

• Pan et al., Measuring Agents in Production (arXiv 2512.04123, submitted December 2025)
• LangChain, State of Agent Engineering (survey of 1,340 respondents, November 18–December 2, 2025; report published February 2026)
• OWASP GenAI Security Project, Top 10 for Agentic Applications 2026 (released December 9, 2025)