An AI Code Review Checklist That Actually Catches Bugs

A concrete checklist for reviewing AI-generated code: spec match, tests that check behavior, hallucinated APIs, error handling, security, and readability.

Most code review checklists were written for humans, and they check for the mistakes humans make: typos, off-by-one errors, forgotten edge cases picked up from years of experience. AI-generated code fails differently. It reads clean, compiles on the first try, and often ships with a passing test suite, which is exactly why it needs a different checklist, not a lighter one. Approving a diff because it looks fluent is how the failure modes below make it to production.

The checklist below is built around six checks that map to the specific ways AI-assisted code goes wrong: drifting outside the requested scope, tests that prove nothing, hallucinated API calls, missing error handling, security gaps, and code nobody can review carefully enough to catch problems in. Each check targets one failure mode, not a vague sense of quality, and each is fast enough to run on every pull request.

Check the Spec Before You Check the Code

• Does the diff touch only the files and functions the spec named, or has it expanded quietly into adjacent code the request never mentioned?
• Does every new dependency, config flag, or environment variable trace back to something the spec actually asked for?
• If an existing function's behavior changed, was that change part of the request, or a side effect of the model "cleaning up" while it was in the file?
• Would a reviewer who only read the spec, not the code, be surprised by anything in the diff?
• Is there a caching layer, retry wrapper, or "helpful" abstraction that nobody requested and that now needs its own review?

Tests That Prove Behavior, Not Mocks

A passing test suite is not proof of anything by itself. AI-generated tests have a specific failure pattern: they mock the function under test, assert that the mock was called with certain arguments, and call that coverage. That kind of test catches a signature change. It does not catch a logic error, because the actual logic never ran.

• Do the tests exercise real inputs and check real outputs, or do they assert that a mocked dependency was called?
• Is there at least one test for a failure path (bad input, downstream timeout, empty result), not only the happy path?
• If the implementation were replaced with a stub returning a hardcoded value, would any test fail?
• Do the tests cover the specific edge case the spec called out by name, not just "general" coverage of the function?

Hallucinated APIs and Phantom Signatures

Language models produce fluent, syntactically valid calls to methods that were deprecated versions ago, functions that belong to a different library with a similar name, or parameters that sound right but never existed in the real signature. This is one of the most common failure modes in AI-generated code and one of the hardest to catch by reading, because the code looks exactly like correct code.

• Does every third-party function call match the version of the library actually pinned in the project, not the version the model was trained on?
• Do argument names, order, and types match the real signature, checked against source or docs, not against what looks plausible?
• Are configuration keys, environment variable names, and CLI flags checked against a working example that already exists in the repo?
• If the code imports something new, does that package actually exist and get maintained, rather than sounding like a plausible name?

Error Handling, Security, and Readability

AI-generated code tends to handle the case it was explicitly asked about and skip everything adjacent to it, including the input validation, permission checks, and secret handling that never appear in a short prompt but are exactly what separates a working demo from something safe to expose to real traffic. Readability belongs in the same check, because code nobody can review quickly gets less scrutiny next time, not more, and that is how the first five checks stop happening.

• What happens on empty input, null, a string where a number was expected, or a request that fails halfway through?
• Is external input (user text, file uploads, third-party API responses) treated as untrusted before it reaches a database query, shell command, or template render?
• Does every new endpoint or function check that the caller is authorized for the specific resource, not just that they are logged in?
• Are secrets, API keys, and credentials read from environment variables or a secrets manager, never hardcoded and never written to a log line?
• If a downstream call fails, does the code fail closed (deny, return an error) or fail open (silently proceed with default behavior)?
• Can you explain what a function does from its name and signature alone, or does the diff need to be split so a reviewer can hold the whole change in their head?

The Checklist at a Glance

Running six checks by hand on every pull request does not scale past the first few weeks, which is why this works better as a gate than as a habit someone has to remember. TLM Forge runs independent multi-agent review against this kind of list before code ships, paired with a spec audit so scope drift gets caught before the diff even exists; see how it works and the idea behind it in spec-driven development. The security half of this checklist, injection, authorization, and secrets handling, is worth a deeper pass on its own; see security review of AI code for that. Some of the scope-drift problem in the first check starts before the diff exists at all: a model that loses project context between sessions has to re-derive conventions every time, and re-deriving badly looks like an unrequested refactor. A private, persistent memory layer such as MemX that carries project context and prior decisions across sessions can reduce how often that happens, though it does not replace the checklist above; even a model with perfect memory of the codebase can still write a test that only checks a mock.