← Back to BlogCode Review

How Much Should You Review AI-Generated Code?

Set review depth by reversibility, reach, sensitivity and detectability, not diff size. Four depths, what to do at each, and commands to flag risk.

Review an AI change as hard as the damage it could do, and let evidence, not the model's confident tone, buy you a lighter pass. Four questions set the depth: can the change be undone, how far does a failure reach, is it in a category that always gets a person, and would you notice if it were wrong. Take the worst answer, not the average.

Disclosure: we build TLM Forge, a process layer for Claude Code, so weigh the product section accordingly. Most of this post applies to any agent and any editor.

The question developers keep asking

A question on Hacker News put it plainly: "do both get the same level of code review? or do some parts of the code get heavier code review than others?" One commenter answered that it depends on what the code is solving: "UI or low risk code I can let the agent do everything. Auth, security, performance? I dig deeper manually or ask agent more about what it has done, why..." In a separate thread about whether companies require human review of AI-generated code, another commenter wrote: "Both my employers do, but it is not genuinely getting human reviewed anyway. I frequently let my Claude approve PRs that I have not even read the title of."

Sources: Hacker News: Do you review AI generated code differently based on where it is in your code?, Hacker News: Ask HN: Does your company require human review of AI-generated code?

These are two individual comments, not measurements. They mark the poles: depth chosen by risk, and a blanket rule that requires review of everything, which the second commenter says is not really happening at either employer. What is missing is a procedure you can run in a minute on a real diff.

A useful published mental model is Birgitta Böckeler's. Writing on martinfowler.com, the Thoughtworks Distinguished Engineer describes risk assessment as a combination of probability, impact and detectability, illustrates the two extremes, and says most situations land somewhere in between. This post fills in the middle. It splits impact into reversibility, reach and sensitive category, keeps detectability, and treats probability as a modifier, because how likely the model is to be wrong is hard to read off a diff.

Sources: Birgitta Böckeler, To vibe or not to vibe (martinfowler.com)

Why one depth for everything fails both ways

Maximum depth on every change is not the standard, even where review is taken seriously. Google's reviewer guide says to look at every line you are assigned, then adds that some code deserves more careful scrutiny than other code. A 2018 study of 9 million reviewed changes at Google found that over 35% modified a single file, the median change was 24 lines, and fewer than 25% had more than one reviewer. Its authors summarize that the majority of changes "are small, have one reviewer and no comments other than the authorization to commit." The data covers 2014 to 2016, before coding agents, so read it as a baseline, not a target.

Sources: Google eng-practices: What to look for in a code review, Sadowski et al., Modern Code Review: A Case Study at Google

A heavier, external approval step on every change has not been shown to buy safety either. DORA's research found no evidence that a more formal, external review process was associated with lower change fail rates, and linked heavyweight approval to larger, less frequent releases and higher failure rates. Its alternative keeps peer review, adds automated testing and monitoring, and includes "ongoing analysis to detect and flag high risk changes early on so that they can be subjected to additional scrutiny." That page covers change approval boards, not agents, but the lesson transfers: flag the risky changes instead of slowing all of them.

Sources: DORA: Streamlining change approval

Amazon's 2015 shareholder letter names the same trap. It separates decisions that are "consequential and irreversible or nearly irreversible", which it calls one-way doors, from reversible two-way doors, and warns that large organizations drift into using the heavyweight process on most decisions. The result it lists is "slowness, unthoughtful risk aversion", among other costs. Review depth is a decision of that kind. Spend the slow process where the door only swings one way.

Sources: Amazon 2015 letter to shareholders

The opposite failure is review that quietly stops. DORA's 2025 report on AI-assisted development says AI adoption "does continue to have a negative relationship with software delivery stability," and ties it to rising change volume landing on weak automated testing, version control practice and feedback loops. It is an association found in survey and qualitative research, so it cannot say whether any one diff is risky. It does fit DORA's warning that acceleration can expose weaknesses downstream, which is a reason to keep your checks when the agent is fast, not to drop them.

Sources: Google Cloud Blog: Announcing the 2025 DORA Report

Four questions that set how much review AI-generated code needs

Answer these from the change, before you read it closely. Score the risk on each one as low, medium or high. A yes on question 3 counts as high.

1. Can it be undone?

Amazon's door image is the sharpest test. A change you can revert in a minute is a two-way door. A migration that drops a column, a command that deletes a resource and a message already sent to customers are one-way doors, whatever the diff looks like. Git reverts the code, not the effect. Claude Code's checkpointing restores files its editing tools changed, and its docs say a file that only Bash commands changed "isn't tracked and stays as it is". A table a command already altered is not a file at all, so a rewind does not reach it. What Claude Code Rewind Cannot Undo covers the gaps.

Sources: Claude Code docs: Checkpointing

2. How far does a failure reach?

A leaf script one person runs and a helper imported by forty modules are different bets at the same line count. Böckeler's sanity checks fit here: whether the code has a high impact radius, meaning many other components or consumers use it, and whether you would ship it if you were on call tonight. For agent work, also count what the change touched beyond the request. Files you never named widen the reach, whatever the reason they were edited.

Sources: Birgitta Böckeler, To vibe or not to vibe (martinfowler.com)

3. Is it in a category that always gets a person?

Some code gets a qualified human whatever the other answers say. Google's guide tells a reviewer who does not feel qualified for part of a change to make sure a qualified reviewer is on it, and names privacy, security, concurrency, accessibility and internationalization. It singles out parallel code because race conditions and deadlocks are "very hard to detect by just running the code." Add the categories where a mistake costs money or trust: authentication and authorization, payments, personal data, secrets and cryptography, infrastructure and permission changes, and migrations.

Sources: Google eng-practices: What to look for in a code review

Security standards make the same move at the application level. OWASP's ASVS defines three verification levels and says an organization should analyze its risks and choose a level "depending on the sensitivity of the application." Once you have your list of categories, Approval Gates for High-Risk AI Code Changes shows how to make a pipeline enforce it instead of relying on memory.

Sources: OWASP ASVS 5.0: What is the ASVS?

4. Would you notice if it were wrong?

Detectability is the question that lets you lower depth honestly. A change that crashes on the first request is cheap to review shallowly. One that quietly writes a wrong total is not. Böckeler lists good tests, a typed language, a stack that makes failures obvious and your own familiarity with the codebase. Google's guide adds a warning that applies double to agent output: tests "do not test themselves," and "a human must ensure that tests are valid." An agent can write the change and the tests that bless it in one pass, so a green suite helps only if you have read what the tests assert. An AI Code Review Checklist That Actually Catches Bugs covers what a test should prove.

Sources: Birgitta Böckeler, To vibe or not to vibe (martinfowler.com), Google eng-practices: What to look for in a code review

Insight

Let the highest score set the depth. A tiny, easy-to-see change that cannot be undone is still a one-way door, and averaging hides exactly that case.

Raise the depth one step when any of these is true, because each one lowers your odds of spotting a mistake or raises the odds that one exists:

  • The request was vague and the agent filled the gaps itself.
  • The diff includes files you did not name in the request.
  • Tests were edited in the same diff as the code they check.
  • The code area is one you could not debug on your own.
  • A new dependency appeared.

Four depths and what you do at each

The names describe what you do, not how long you spend. Take your highest score and pick the row whose typical change fits it.

DepthTypical changeWhat you doWhat you want to see first
Depth 0: GlanceDocs, comments, copy, a throwaway scriptRead the file list and the summary. Confirm nothing outside the request changed.CI green and nothing on your risky-path list
Depth 1: ReadLeaf code that is easy to revert and fails loudlyRead the whole diff once. Run it once yourself.A test that would fail if the change were wrong
Depth 2: TraceShared code, or a change that could fail quietlyRead every line, open the callers, and read the tests as closely as the code.Evidence from the agent: the commands it ran and their output, plus the wider suite green
Depth 3: Design first, then line by lineCannot be undone, or touches auth, money, personal data or permissionsApprove the plan before any code exists. Then read every line with a second person who knows the area, and run it somewhere real.A written plan, a rollout you can stop, and a reviewer with the right knowledge

Depth 2 depends on evidence you ask for. Anthropic's best-practices page says to ask Claude for the command it ran and its output, and that "All tests pass" on its own tells you very little. It also describes a review pass by someone who did not write the code: in the terminal, /code-review reviews the diff in a fresh subagent that never saw your conversation.

Sources: Claude Code docs: Best practices

Depth 3 starts before the code. For a one-way door the diff is the wrong place to begin, because a plan is cheap to reject and a finished change is expensive to unwind. Claude Code's plan mode exists for this: it tells Claude to "research and propose changes without making them", and edits stay blocked until you approve the plan, except in interactive terminal sessions where bypass permissions are available. I Don't Understand the Code My Agent Wrote explains why approving the design beats reading a 900-line diff cold.

Sources: Claude Code docs: Choose a permission mode (plan mode)

Size sets how long it takes, not which depth

Line count still matters, as a cost on your attention. Google's guide says small changes are reviewed more thoroughly, because in large ones important points sometimes get missed or dropped. It gives 100 lines as usually reasonable and 1000 as usually too large, leaves the call to the reviewer, and counts the number of files a change spans as part of its size. For agent work, a big Depth 3 change is a reason to split it so Depth 3 stays affordable, while a three-line edit to token validation is Depth 3 at any size. Stop Your AI Agent Writing 1,000-Line PRs covers getting smaller pieces from the agent.

Sources: Google eng-practices: Small CLs

What earns a lighter pass

Lowering depth is a claim about evidence. Before you drop a step, you should be able to name what would have caught the mistake instead:

  • A test that fails when the behavior breaks, and that you have read.
  • Types or a compiler that reject the likely mistakes.
  • A failure you would see within minutes, such as a crash, an alert or a failing health check.
  • A revert or a flag flip you can perform in minutes.
  • Enough familiarity with the code to notice a wrong move.

The agent sounding sure is not on that list. In a user study first posted in 2022, 47 participants did security-related programming tasks, and those with an AI assistant built on OpenAI's codex-davinci-002 wrote significantly less secure code than those without one. They were also more likely to believe their code was secure. It tested one assistant on security tasks, so it cannot say how agents built on other models behave. It does show that, for those participants, confidence and security moved in opposite directions.

Sources: Perry et al., Do Users Write More Insecure Code with AI Assistants? (arXiv)

An AI reviewer can be one input, and it has its own depth setting. Claude Code's docs say a /code-review effort level trades coverage for confidence: at low it reports the findings it is most confident in, and from medium through max it broadens coverage. The hosted Code Review product has Quick, Standard and Deep review levels, and its docs give Quick for a routine codebase and Deep for a critical one as an example. Its page also warns that a review with no findings does not mean the code is error-free or secure, and says a review works from the diff and surrounding code without building or running it. That limit is one reason Depth 1 includes running the change yourself.

Sources: Claude Code docs: Review locally with /code-review, Claude Code docs: Code Review

When you cannot review properly before you must ship, defer depth and never drop it. Google's guide defines an emergency as a small change such as one that fixes a bug significantly affecting users in production or closes a major security hole. It lets the reviewer weigh speed and correctness above everything else, then says to give the change a more thorough review afterward. Wanting to launch a week sooner, with no hard deadline behind it, is on its list of non-emergencies.

Sources: Google eng-practices: Emergencies

Commands that flag the risky part of a diff

Depth 3 triggers often show up in the file list before you read a line. These two commands list changed files in risky places and added lines that do irreversible things. Replace origin/main with your base branch, and add your own directory names. Both compare commits, so commit the agent's work to a branch first: uncommitted edits and new untracked files do not appear.

  • Risky paths: git diff --name-only origin/main...HEAD | grep -Ei '(^|/)(auth|authn|authz|authentication|authorization|login|session|permissions?|passwords?|payments?|billing|checkout|migrations?|infra|terraform|iam|secrets?|crypto)(/|[._-]|$)|\.sql$|\.tf$|(^|/)\.github/workflows/|(^|/)dockerfile'
  • Irreversible lines: git diff -U0 origin/main...HEAD | grep -E '^\+[^+]' | grep -Ei 'drop (table|column|index)|truncate|delete from|alter table|rm -rf'

We ran both on a scratch repository with git 2.54.0, using BSD grep and ugrep. A branch touching an auth file, a migration, a workflow and a Terraform file printed those four paths and the ALTER TABLE line. A branch changing only a README and a button component printed nothing. An empty result is not a clean bill: it only means your naming patterns did not match, so a payments module in a folder called ledger would pass straight through. Use it to raise the depth, never to lower it.

Pro Tip

Ask the agent for an index into the diff, then check it against the diff: "Before I review, list every file you changed that I did not name, anything you ran or changed that git cannot undo, every auth, payment, personal-data or permission path you touched, which tests would fail if this change were wrong and which only use mocks, and what you did not verify." Treat the answer as a table of contents, not a verdict. If it reports nothing risky and the file list disagrees, the file list wins.

Where TLM Forge fits

TLM Forge sizes each task Light, Medium or Deep before it builds anything. Light gets no review agents, so the independent review there is whatever you bring. Medium and Deep show a design for your OK before building, which is the cheapest place to apply Depth 3 to an irreversible change. Deep gets a spec audit and a larger final review wave.

The limit is the sizing itself. The model makes that call by following the plugin's own rule, not the four questions above, so it can size a change lighter than you would. You can tell it to go deeper or lighter, and the Depth 3 list above should stay yours. None of it replaces a person who knows the database or the payment flow.

Frequently asked questions

01Do I need to review every line of AI-generated code?

Read every line by default. Google's review guide says to, while letting reviewers scan data files and generated code and asking for more scrutiny on some code. Glancing at the file list is reasonable only for docs, comments or a throwaway script. Anything irreversible or sensitive gets every line, read slowly.

02Does AI-generated code need more review than human code?

One study is not a general answer. In a 47-person study first posted in 2022, people using an AI assistant wrote less secure code and were more likely to believe it was secure. A sure-sounding answer is not a reason to review less. Set depth by the risk of the change, whoever wrote it.

03Is it safe to skip review on small AI changes?

Size is the wrong test. A three-line change to a permission check can do more damage than a large rename. Glance at the file list, confirm nothing outside the request changed, and check that a failing test would catch a mistake. Anything irreversible or sensitive gets read whatever its size.

04What should I check first when reviewing an AI pull request?

Start with the file list, not the code. Look for files you did not ask about, migrations, workflow or permission changes, and edits to tests. Those tell you the depth before you read a line. Then read the tests as carefully as the code, because a test that cannot fail proves nothing.

05Can an AI code reviewer replace a human reviewer?

Not for the riskiest changes. Anthropic's Code Review docs say an AI review may miss bugs and that a review with no findings does not mean the code is error-free or secure. Use automated review as a first pass, and keep a person on anything irreversible or sensitive.

Ship AI-written code you can trust

TLM Forge is the missing process layer for Claude Code: a spec audit, independent multi-agent review, enforced TDD, and an adversarial red-team gate.

Get TLM Forge