How to Write Acceptance Criteria an AI Agent Can Verify

Quick answer
An acceptance criterion is a condition a change must satisfy, written so that someone else can check it without reading the implementation.
Use Given/When/Then or an equivalent shape: state the setup, the action, and the observable result.
One criterion should describe one behaviour, including the failure and boundary cases, not a sequence of implementation steps.
Replace adjectives with thresholds: not fast but under 300 ms at p95 for 50 concurrent requests; not secure but rejects a request without a valid tenant claim.
Each criterion should map to at least one automated test, and every test should trace back to a criterion.

What makes an acceptance criterion verifiable?

It names a starting state, an action, and an observable outcome. Given an authenticated user with three orders in the range; when the export endpoint is called; then the response contains three data rows and a header. Anyone can run that and see whether it passed, without interpreting what the code was supposed to do.

The outcome must be observable from outside the implementation: a response, a database row, an emitted event, an error code. A criterion such as the service uses a queue internally describes a design choice, not a behaviour, and cannot be verified without reading the code — which is exactly what acceptance criteria are meant to avoid.

Why do AI agents need criteria phrased differently?

Because an agent optimises for the literal text. If the criterion says the endpoint should be fast, the agent will choose something that looks fast and declare success; there is no shared threshold to miss. If it says handle errors gracefully, the agent will invent a definition of graceful and implement that.

The fix is to move every hidden decision into the criterion: the status code, the retry behaviour, the maximum size, the time bound, the exact field names. This is not pedantry aimed at the agent; it is the same precision a human reviewer would need to agree on what done means.

Weak criteria                          Verifiable criteria
-------------------------------------  ------------------------------------------------
Export should be fast.                 p95 under 2 s for 10k rows on the stated host.
Handle invalid dates.                  from > to returns 400 with code INVALID_RANGE.
Secure by default.                     A token for tenant A returns no tenant B rows.
Good error messages.                   429 body is {code: RATE_LIMITED, retry_after_s: 60}.
Works with large data.                100,001 rows returns 400 RANGE_TOO_LARGE, no partial file.

How many criteria does a feature need?

Three to seven for a typical feature. Fewer usually means the happy path is specified and the failures are left to the implementer; many more usually means the criteria have become a test plan or a design document, which belongs elsewhere.

Cover four categories: the happy path, the boundary, the failure, and the permission case. Boundary covers empty results, maximum size, and off-by-one dates. Failure covers invalid input and dependency errors. Permission covers the cross-tenant or unauthenticated case, which is the one generated code most often omits.

How do you connect criteria to tests?

Name the test after the criterion, or tag it with the criterion’s identifier. Then a failing test points back to a requirement, and a requirement with no test is visible. This traceability is what stops the suite from drifting into a description of the implementation.

Review the mapping before implementation, not after. If a criterion cannot be tested as written, it is either unverifiable or the system lacks an observable boundary. Both are useful findings at spec time, and both are expensive to discover when the feature is already built.

Where this bites vibecoders
A prompt like build a CSV export with good error handling gives the assistant no way to be wrong, which means no way to be right either. Five criteria with exact status codes and row counts turn an open-ended generation into a checkable task, and the resulting tests can fail for a reason you chose in advance.

Where AI coding assistants get this wrong

Using adjectives such as fast, secure, robust, or user-friendly without a threshold.
Specifying implementation steps instead of observable outcomes.
Covering only the happy path and leaving failures to the implementer.
Writing criteria that cannot be tested without reading the code.
Combining several behaviours into one criterion so a partial pass looks like a pass.
Writing the criteria after the implementation to match what was built.

Checklist

State the setup, action, and observable result for each criterion.
Replace every adjective with a number or an exact value.
Include empty, boundary, failure, and permission cases.
Keep one behaviour per criterion.
Map each criterion to at least one test.
Review the mapping before implementation starts.

FAQ

Should acceptance criteria use Given/When/Then?

It is a reliable default because it forces a setup, an action, and an outcome. A numbered list with the same three elements works equally well. The format matters less than the presence of an observable outcome and a concrete starting state.

How are acceptance criteria different from a definition of done?

Definition of done is a standing checklist for every change, such as tests pass and docs updated. Acceptance criteria are specific to this feature and describe its behaviour. A feature can satisfy the definition of done and still fail its acceptance criteria, which is why both exist.

Can an AI agent write its own acceptance criteria?

It can draft them, and that draft is useful as a starting point. It should not be the final authority, because the agent will infer the same assumptions it would have implemented. A person must confirm that the criteria describe the intended product, especially the failure and permission cases.

What if a criterion is hard to test automatically?

Either make the behaviour observable through an interface you can test, or accept a manual check and say so explicitly. Do not quietly drop it, because an untested criterion becomes an assumption. A manual check with a named owner is honest; a criterion nobody verifies is decoration.