What done means for an agent project

"We'll know it when we see it" is an expensive phrase in any software project. In agent projects it is worse, because agents fail charismatically. A wrong answer delivered fluently looks a lot like a right one, and a demo that worked three times looks a lot like a system that works.

Agent projects that thrash usually share a root cause: nobody wrote down what done means before building started. So the target moves. Each new demo surfaces a new concern, each concern becomes a new requirement, and the team never gets to say "finished."

The fix is a one-page definition of done, agreed before the first line of code and revisited only deliberately. Below is the template we use in BDArgentumAI engagements, with notes on filling in each section.

The template

1. Outcome

Who benefits, and what changes for them?

Write it in terms of the user's work, not the technology. "Support staff spend less time on routine billing questions" is an outcome. "Deploy a billing agent" is not.

Include the baseline you will compare against, even a rough one: how long the task takes today, how often it goes wrong, what it costs.

2. Scope

Which tasks, which users, which actions?

List the task types in scope and, just as important, those out of scope. Name the actions the agent may take and those it may only propose. Scope creep in agent work often arrives as "could it also just…".

3. Quality thresholds

How will we measure good enough?

Reference a specific evaluation set and state the thresholds:

• Hard failures (policy violations, forbidden actions, unsupported claims on grounded tasks): zero tolerance.
• Soft quality score on the evaluation set: at or above an agreed level, and stable across the last several changes.
• Human review: a stated sample size and minimum acceptance rate during the first weeks of use.

If the evaluation set does not exist yet, building it is the first deliverable.

4. Controls

What must be present and proven?

At minimum: a step cap, approval on consequential actions, per-run and daily spending limits, and an off switch someone has actually used in a rehearsal. Present is not enough; each control needs evidence that it works.

5. Operability

Can someone other than the builders run it?

A named owner. An on-call arrangement. A runbook covering how to disable the agent, roll back a change, and respond to a provider outage. Trace retention long enough to investigate a complaint.

6. Handoff

What does the receiving team get?

Documentation, a decision log explaining why the system is built the way it is, and an internal lead who has been trained and has operated the system while the builders were still around to help.

7. Sign-off

Who decides?

Name the person who can accept the system as done and the evidence they will review. This is the go/no-go decision, written down in advance.

False finish lines

It helps to list the false finish lines explicitly in the document, so nobody mistakes one for the real thing under deadline pressure:

• An executive was impressed by the demo.
• A new model version came out and "seems better."
• Three hand-picked prompts produced great results.
• The team is tired of the project.
• The agent works, but only its author can explain a failure.

How done changes as autonomy grows

A definition of done is tied to a level of autonomy. When the team later proposes that the agent send email without approval, or take on a new task type, that is new scope, and it needs updated thresholds and controls. A reasonable rule: each increase in autonomy tightens the quality thresholds for the affected tasks and adds at least one new control or test.

When the definition needs to change

Sometimes the first definition is simply wrong: a threshold proves unreachable for good reasons, or a user need turns out to be different from what everyone assumed. Changing it is fine. Changing it quietly is not. Record what changed, why, and who agreed, in the same way you would record a change to a contract. A definition of done that moves only through visible, signed-off edits still does its job.

Keeping the document alive

Store the definition of done in the repository beside the code, review edits to it like code, and keep it next to the name of the person who owns the off switch. When people disagree about whether the project is finished, the document settles it. When it cannot, that is a sign it needs a clearer line, which is a much smaller problem than a project that never ends.