How to Test AI-Generated Code (Before It Reaches Production)

Quick answer

• Test AI-generated code more strictly, not less. The technique is identical; the bar for adequate evidence is higher.
• Start with boundaries and error paths: validation, auth, nulls, external failures, money, dates. Happy paths are usually right; edges are where it guesses.
• Treat generated tests as scaffolding. A test that mirrors the implementation passes against broken code by construction.
• Prove your tests work by mutation: break the code on purpose and confirm a test fails.

How do you test code you did not write?

The usual way a person verifies code is by reading it against the intent they formed while writing it. That shortcut does not exist for generated code, because you never held the intent — you described it in a prompt and received something back. You cannot spot a mismatch between “what I meant” and “what the code does” by reading, because you cannot check the code against a mental model you never had.

That is the whole problem, and the answer is that tests become your primary evidence rather than your safety net. You are not testing to prevent regressions in a change you understand; you are testing to answer a question you genuinely cannot answer by inspection: does this do what I asked for?

This reframing matters because it changes what you write. Code you wrote needs tests that protect future changes. Code you did not write needs tests that demonstrate present correctness, especially at the boundaries where the model was reasoning by pattern rather than by specification.

Where this bites vibecoders

The specific way this bites vibecoders is the green checkmark. An assistant writes the feature, then writes its tests, the suite goes green, and that is treated as verification. But tests generated alongside an implementation tend to encode the implementation’s actual behaviour rather than the requirement, so they pass by construction — including when the implementation is wrong. The suite is green and the feature is broken in a way nothing in the repository can tell you about. Mutation testing is what breaks that illusion.

What should you test first?

Not everything equally. When time is limited, generated code has a predictable weakness profile, and the highest-value tests are at the edges.

1. Authorization and ownership checks. The most common serious defect in generated code is a route that verifies the user is logged in and then loads a record by an ID from the request. The tests must confirm that user A cannot read or modify user B’s data — see What Is Broken Access Control (IDOR)?.

2. Input validation and empty states. Null, empty string, empty list, zero, negative, missing field, wrong type. Generated code routinely handles the populated case and assumes the rest.

3. External service failures. What happens when the payment provider times out, returns a 500, or returns an unexpected shape? Generated code usually assumes success. The unhappy path is where the damage is.

4. Anything involving money, dates, or units. Rounding, timezone boundaries, currency, and daylight-saving transitions. These produce silent wrong answers rather than errors.

5. Idempotency and retries. If an operation charges a card or sends an email, it must be safe to run twice — see What Is Idempotency.

Integration boundaries and the happy path come after these, not before. The instinct to test that the main flow works first is inherited from TDD, where you build a test at a time; for reviewing a completed implementation, the edges are where the risk actually lives.

How do you write tests you can trust?

Test the requirement, not the implementation. The failure mode of a generated test is that it asserts what the code does. Ask of every assertion: if the requirement changed, would this test fail? If not, it is testing the implementation and will happily pass against wrong behaviour.

Write the test before you read the implementation. Read the spec and the tests first, then the code. Reading the code first anchors you to what it does, which is precisely the thing you are trying to test independently.

Use the framework’s real test runner. Generated snippets sometimes contain plausible-looking but non-existent helpers. Run them, and confirm the count of executed tests is what you expect — a suite reporting “0 passed” with no failures is a common silent no-op.

import { describe, it, expect } from "vitest";
import { chargeCard } from "./payments.js";

describe("chargeCard", () => {
  it("does not charge when the order total is zero", async () => {
    // A requirement, not a restatement of the code.
    await expect(chargeCard({ orderId: "o1", amount: 0 }))
      .resolves.toMatchObject({ charged: false });
  });

  it("refuses to charge more than the order total", async () => {
    // A boundary the generated code plausibly got wrong.
    await expect(chargeCard({ orderId: "o1", amount: 150, total: 100 }))
      .rejects.toThrow(/exceeds/i);
  });

  it("charges exactly once when called twice with the same idempotency key", async () => {
    const key = "idem-123";
    await chargeCard({ orderId: "o1", amount: 50, idempotencyKey: key });
    await chargeCard({ orderId: "o1", amount: 50, idempotencyKey: key });
    expect(await countCharges("o1")).toBe(1);
  });
});

Note what each test asserts: a rule, stated in the domain’s language. If someone changes the implementation to a different-but-similar approach, these should still pass — and if someone breaks the rule, they should fail.

How do I know my tests are actually testing something?

Run a mutation check: deliberately introduce a small bug and confirm a test catches it. This is the only reliable way to know a suite has teeth, and it takes two minutes.

// Original
return amount > 0;

// Mutant — should make a test fail
return amount >= 0;

If the whole suite still passes, the tests are not asserting on that behaviour. Common findings: assertions that check a value was returned but not what it was, mocks so thorough they only verify the mock, and tests that assert on a constant.

This matters disproportionately for generated tests, because the failure mode is systematic rather than occasional. A human writing tests occasionally writes a weak one; an assistant asked for “good test coverage” will write a whole file of them, because mirroring the implementation is the path of least resistance and it produces an impressive-looking green suite.

What about code coverage?

Treat coverage percentage as a floor, not a goal. Generated code routinely reaches high line coverage with tests that assert almost nothing, which is why a coverage number tells you very little on its own.

Two adjustments make it useful:

• Look at branch coverage, not line coverage. Line coverage counts whether a line ran; branch coverage counts whether both outcomes of a condition were exercised. Error paths are branches, and error paths are where the risk is.
• Check that your important code is covered, not that most of it is. Cover the authentication path, the authorization check, the money path, and the validation. A file at 40% that contains all four is safer than a file at 95% that covers none of them.

The deeper point is that you cannot get this evidence by asking an assistant for “tests with good coverage”. It will produce coverage. What it cannot produce is the knowledge of which behaviours mattered, because you did not state them.

Where AI coding assistants get this wrong

• Generating tests from the implementation, so they pass by construction and prove nothing.
• Testing the happy path only, because that is what the visible example demonstrated.
• Asserting on call counts and mocks rather than on observable behaviour, which verifies the test’s own setup.
• Mocking the very logic under test, including the function being tested in its own test.
• Using non-existent or renamed helpers, producing tests that silently never run.
• Claiming coverage as evidence of correctness, which conflates “executed” with “verified”.
• Writing tests that encode a bug as expected behaviour, freezing the mistake in place.
• Skipping the negative cases — the empty list, the null, the permission denied, the upstream timeout.

Checklist

• Read the spec and the generated tests before the implementation, so you are not anchored to it.
• Start tests at boundaries: validation, auth, empty states, external failures.
• Cover authorization explicitly — user A must not reach user B’s data.
• Cover money, dates, and unit conversions at their edges.
• Assert on rules stated in domain language, not on implementation details.
• Run the suite and check the executed test count is what you expected.
• Confirm 0 passed with no failures is not silently hiding everything.
• Run at least one mutation check to prove the suite has teeth.
• Prefer branch coverage to line coverage.
• Confirm the risky code is covered, not merely that the percentage is high.
• Make operations that charge or send idempotent.
• Keep the tests in CI, not just on your machine.

FAQ

Should AI-generated code be tested differently from code a person wrote?

Yes, and the difference is about where you spend effort, not about different techniques. Because you did not write it, you cannot review it by reading for intent — you did not form the intent — so tests become your primary evidence that it does what the spec requires. The bar for what counts as adequate coverage is higher, not lower, and coverage percentage alone is a poor proxy for it.

What should I test first in AI-generated code?

Start with the boundaries and the error paths, because that is where generated code is weakest. Validation, authentication and authorization checks, empty and null inputs, external service failures, and anything involving money or dates. Happy-path logic is usually correct; the edges are where the assistant produced confident, plausible, untested code.

Are AI-generated tests trustworthy?

Only as a starting point. An assistant will write tests that mirror the implementation rather than the requirement, so they pass against buggy code by construction. Treat generated tests as scaffolding: read each assertion and ask whether it would fail if the code were wrong. A test that cannot fail is worse than no test, because it creates confidence you have not earned.

How do I know my tests are actually testing something?

Check them by mutation: deliberately break the code in a small way and confirm a test fails. If your whole suite still passes after you introduce a bug, the tests are not asserting on the behaviour you think they are. This check takes two minutes and catches the most common failure in generated test suites, which assert on things that cannot change.

What coverage number should I aim for?

No number is a target in itself. Cover the authentication path, the authorization check, anything handling money, and the validation logic, and accept that this may be a minority of your lines. High line coverage with no error-path coverage is the false-confidence case, and it is the most common result of asking an assistant for good coverage.

Should I ask the AI to write the tests too?

Use it for scaffolding — generating the file structure, the setup, the fixtures, and the first pass at cases — then review every assertion yourself. The value is in the volume of starting points, not in the assertions themselves. If you accept the assertions without reading them, you have added a green checkmark rather than evidence.

Related topics

• How to Write Your First Unit Test
• What Is Test-Driven Development (TDD)?
• How to Review AI-Generated Code Like a Senior Engineer
• Unit vs Integration vs End-to-End Tests: What’s the Difference?
• What Is AI Code Validation?

Sources

• Martin Fowler — Mutation Testing
• Martin Fowler — Test Double
• Google Testing Blog — Code Coverage Best Practices
• OWASP Web Security Testing Guide
• Stryker Mutator documentation