A demo is designed to convince. A production readiness review is designed to decide. Agent projects get into trouble when the first is allowed to substitute for the second, because a fluent demo is exceptionally persuasive and tells you almost nothing about how the system behaves on its five-hundredth run.
The remedy is a written go/no-go review held before launch: a meeting with a fixed agenda, required evidence, and a recorded decision. It is the discipline we teach in the Agentic Production Series and apply when hardening a prototype for production. Here is how to run one.
Who attends
Keep it small: the engineering lead, the product owner, someone from security, the person who will be on call, and a business owner who can accept risk. If nobody in the room can accept risk on behalf of the business, the meeting cannot end in "go."
The evidence pack
Circulate these at least two days before the meeting. Reviewing evidence live wastes the room.
- One-page architecture: control flow, tools, data sources, and where model decisions happen.
- Eval report: golden-set results, with the hard-fail count and soft-score trends across recent changes.
- Two traces: one successful run end to end, and one where a guardrail fired, such as a refused injection or a blocked write.
- Guardrail map: controls listed against the applicable categories of the OWASP Top 10 for Agentic Applications.
- Cost and latency profile: the per-run distribution and the configured limits.
- Runbook: including how to disable the agent, roll back, and handle a model provider outage.
- Approval RACI: who approves which actions, and their response target.
If the eval report is missing, the review should stop there. LangChain's State of Agent Engineering survey found that only about half of organizations run offline evals, and a team without them is asking the room to judge quality by anecdote.
A 75-minute agenda
| Time | Topic | Question the room must answer |
|---|---|---|
| 0–10 | Outcome and blast radius | What does this do for users, and what is the worst thing it could do? |
| 10–20 | Control flow | Where does the model decide? What is the step cap, and what happens when it is reached? |
| 20–35 | Eval evidence | Are hard fails at zero? Are soft scores stable or improving? What are we not testing yet? |
| 35–45 | Guardrails | Which OWASP risks apply, and which control and test covers each? |
| 45–55 | Cost, latency, stop mechanism | What are the limits? When was the off switch last rehearsed, and how long did it take? |
| 55–65 | Operability | Who is on call? Could someone who did not build this follow the runbook? |
| 65–75 | Decision | Go, no-go, or go with conditions, recorded with owners and dates. |
Three possible outcomes
Go. The evidence supports launch at the proposed scope. Record the scope precisely: which users, which actions, what volume.
Go with conditions. Launch is acceptable with named watches: a limited user group, a lower spending cap, approval on an action that may later be relaxed, or a metric reviewed weekly. Every condition has an owner and a review date.
No-go. Something essential is missing. Write down exactly what, so the next review is about closing gaps rather than relitigating the demo.
A no-go is not a failure of the team. It is the review doing its job, catching an expensive problem at the cheapest possible moment.
A decision memo template
Agent: <name> Review date: <date>
Decision: GO | GO WITH CONDITIONS | NO-GO
Scope: <users, actions, volume>
Evidence reviewed: eval report v<n>, traces <ids>, runbook v<n>
Conditions (if any): <condition> — owner — review date
Open risks accepted: <risk> — accepted by <name>
Next review: <date or trigger>Keep these memos together. Six months from now, when someone asks why the agent may send email without approval, the answer should be a document, not a recollection.
Re-reviews
A go decision covers a specific scope, not the agent forever. Schedule a fresh review when any of these change: the model or provider, the set of tools with side effects, the user population, or the approval policy. A short re-review against the same agenda usually takes half the time of the first, because the evidence pack already exists and only the deltas need discussion.
Evidence before enthusiasm
The hardest part is not the agenda. It is agreeing in advance that evidence outranks enthusiasm, so that a charismatic demo cannot outvote a missing kill switch. Leaders set that tone by asking for the eval report first and the demo last.
Sources cited
- LangChain, State of Agent Engineering (survey of 1,340 respondents, November 18–December 2, 2025; report published February 2026)
- OWASP GenAI Security Project, Top 10 for Agentic Applications 2026 (released December 9, 2025)
Keep reading
Agentic AIA season of agentic production: Fall 2026
Since mid-July this blog has published twenty pieces on putting agents into production: workflows before autonomy, evals, MCP, context, approvals, guardrails, cost, latency, retrieval, and injection. Mid-October is a goo
LeadershipWhat done means for an agent project
"We'll know it when we see it" is an expensive phrase in any software project. In agent projects it is worse, because agents fail charismatically. A wrong answer delivered fluently looks a lot like a right one, and a dem
Agentic AIPrompt injection and goal hijack
Any agent that reads email, tickets, documents, or web pages will eventually read a sentence written to manipulate it. Sometimes it is crude: "ignore your previous instructions." Sometimes it is subtle: a support ticket