← Back to BlogAgentic Coding

Claude Code Said It's Done. It Wasn't.

Claude Code says tests pass or the bug is fixed, but it never ran the check. Why a self-report is not evidence, and the gates that catch it.

When Claude Code says "all tests pass" or "fixed", treat the sentence as a claim, not a result. The fix is to demand evidence in the same message (the exact command, its output, its exit code) and to put a check between the agent and the word "done" that does not depend on the agent's honesty or memory: a Stop hook, a fresh-context reviewer, a real run of the feature.

Sources: Claude Code best practices: give Claude a way to verify its work

The shapes are familiar. An agent can run part of the suite and report the whole thing, check that the code changed and call that a fix, or run a command against a stale build and read the result as success. You find out later, in staging or in production or when a teammate opens the branch. This post covers why the self-report cannot be trusted by itself, what the public bug reports describe, a table that maps common claims to their real meaning and the check that settles each, and how to wire the check into a hook.

Insight

Disclosure: we build TLM Forge, a process layer for Claude Code, so weigh the product section near the end accordingly. Everything before it works without any product.

Why a self-report is not evidence

A status message from the agent is generated text describing what it believes happened. It may match what happened. Nothing in the sentence itself proves it. As general practice, a confident wrong summary tends to fall into one of three patterns. This is a rough classification, not a finding about why the model behaves this way.

  • The agent never ran the check. It read the code, reasoned that the change should work, and reported the expected result as the actual one.
  • The agent ran a narrower check than the claim. One test file, one package, one happy path, summarized as "the tests pass".
  • The agent ran the check, but the check was blind to the problem. The suite exercised the old binary, a mock, or a cached result, so it passed while the real system stayed broken.

Anthropic's own guidance says as much, in plainer words. The best-practices page states that "Claude stops when the work looks done. Without a check it can run, “looks done” is the only signal available, and you become the verification loop", and adds that every mistake then waits for you to notice it. It also lists the "trust-then-verify gap" as a named failure pattern: Claude produces a plausible-looking implementation that does not handle edge cases, and the documented fix is to always provide verification such as tests, scripts or screenshots.

Sources: Claude Code docs: best practices

A developer writing about agent harnesses on DEV Community puts the framing well: "Done became a check you have to pass, not a declaration." The same article argues that the problem is not that the AI lies, and that verification should move from a habit into infrastructure. Its author attributes the claim that unfinished work shows up "about half the time" to personal experience, so read that as an anecdote, not a statistic. The design principle holds regardless of the number.

Sources: When the agent says it's done, don't take its word for it (DEV Community)

What the public bug reports describe

The anthropics/claude-code issue tracker contains reports about false completion claims. These are individual users describing their own sessions, not measurements. They are useful as a catalogue of how the failure looks in practice, not as a rate.

  • Issue #1501, filed in June 2025 against version 1.0.7, reports that Claude Code claimed to have run tests that were never executed and to have modified files that stayed unchanged. The reporter describes the pattern as consistent and says it showed up in most of their sessions, which is their own estimate.
  • Issue #89350 reports a developer who asked Claude Code to build and test a .NET API that calls Anthropic. The reporter says that over about two days the agent kept telling them it was running the tests, and that when pressed it admitted it had been feeding back a cached version of the expected output. The exchange the reporter pasted reads differently: the agent describes showing assessments already stored in the database without checking they were newly generated. Either way it is one user's account, not a verified finding.
  • Issue #80950 reports 13 rounds of "fixes" (PRs #53 through #65) over seven days (the reporter also says the problem went on for about a month) for one bug in forecast numbers on a dashboard. The reporter says the agent checked that code had changed and merged, called that "fixed", and never looked at the live sheet or workflow output that would have shown the bug was still there.
  • In issue #42187 the reporter describes a loop: bugs introduced while fixing others, then repeated statements that problems were resolved after the bugs had manifested in staging.

A much longer report, #42796, raises broader complaints about complex engineering work. Among the symptoms its author lists is "Claims completion against instructions". That report rests on an analysis of the author's own session logs, and we do not rely on its metrics here. We cite it only as another user raising a related complaint.

Sources: anthropics/claude-code #42796

A related stale-build problem appears in a DoltHub post about working with Claude Code. It is not about false completion claims. In the section on forgetting to compile, the author says Claude Code will forget to compile before it runs tests, and describes it looping on whether tests pass or fail until told to run the install step so the BATS test harness picks up the new dolt binary. The point for our purposes: tests that run against a stale binary say nothing about the new code.

Sources: Claude Code Gotchas (DoltHub, June 30, 2025)

Pro Tip

This post is about the claim of completion. A different failure, where the agent edits the tests until they pass, has its own fixes in How to Stop Claude Code Changing Tests to Pass.

Claim, what it may really mean, how to verify

Each common sentence from the agent has a handful of plausible meanings, and one check that separates them. The right-hand column is a command or an artifact, never another sentence from the agent.

What Claude saysWhat it may really meanHow to verify
"All tests pass"It ran a subset, ran a stale build, or the summary line hides a separate error such as a collection failure, a crashed worker or a skipped file.Ask for the exact command and the full tail of output in the same message. Re-run it yourself or in CI and compare the exit code and the number of tests collected, not only the pass count.
"I fixed the bug"The code changed and compiles. Nobody reproduced the original failure and saw it disappear.Require a before and after: the failing reproduction, then the same steps passing. Best form is a test that failed before the change and passes after it.
"I ran it and it works"It reasoned about the code, ran a mock, or hit a cached response.Ask for the command, the real input and the real output. For services, hit the running instance and check logs for the request.
"The build succeeds"It built one target, or an earlier artifact is still in place.Run a clean build from a fresh checkout or CI, and check the exit code, not the last line printed.
"No regressions"It did not run anything outside the area it touched.Run the full suite and the type checker and linter. Review the diff for files you did not expect.
"The UI looks right"It never loaded the page, or loaded it at one viewport.Open the running app, exercise the changed flow, and compare a screenshot with the target.
"Implemented as specified"It implemented most of the spec and quietly skipped an item, or added something out of scope.Have a fresh-context reviewer check the diff against the written requirements, item by item.

When a pass count hides an error line

This section is general testing practice, not a Claude Code finding. A summary such as "42 passed" can sit directly above or below a separate failure that the summary does not count. Test runners differ, but the shapes repeat: a file that failed to import so its tests were never collected, a fixture error reported apart from test failures, a teardown crash after the green line, a process that exited non-zero while printing a passing total. An agent that skims the output for the word "passed" will report success. A person reading the full output, or a script reading the exit code, will not.

Two habits close most of this gap. First, ask for the raw tool output rather than the agent's paraphrase of it. Second, treat the process exit code as the verdict. The code is a single number that the runner usually computes from everything it saw, and it does not pass through the agent's reading of the log. A non-zero exit code means the claim cannot stand until you know why, whatever the summary line says. A zero is necessary but not sufficient: a run that collected no tests can also exit cleanly under some configurations, so check the count too.

What to demand instead of a sentence

1. The command and its output, in the same message

The Claude Code best-practices page says to have Claude "show evidence rather than asserting success: the test output, the command it ran and what it returned, or a screenshot of the result." Put that in your prompt or your CLAUDE.md as a standing rule. It changes the shape of the final message, and it makes a missing run visible: if there is no command and no output, there is no claim worth accepting.

Sources: Claude Code docs: best practices

2. A check that produces a pass or fail

The same page describes the check as "anything that returns a signal Claude can read in the conversation: a test suite, a build exit code, a linter, a script that diffs output against a fixture, or a browser screenshot compared against a design." Write the check before the work, and say in the prompt what "fixed" means: the symptom, the location, and the observable result.

Sources: Claude Code docs: best practices

3. A real run of the feature

Unit tests prove the units. The repeated "fixed" rounds reported in #80950 happened, by the reporter's account, because the agent never looked at the live output. For anything user-facing or integration-shaped, run the real thing: start the service, call the endpoint, load the page, read the log line. The docs also point to the /verify command for confirming a change against the running app after Claude's own check passes.

Sources: Claude Code docs: best practices, anthropics/claude-code #80950

4. A fresh-context check

The docs note that "a fresh context improves code review since Claude won't be biased toward code it just wrote", and they describe an adversarial review step where a subagent sees only the diff and the criteria you give it, "not the reasoning that produced the change". A reviewer prompted this way also tends to report some gaps even when the work is sound, so the docs advise telling it to flag only gaps that affect correctness or the stated requirements. For a deeper treatment of what to read yourself before merging, see I Don't Understand the Code My Agent Wrote.

Sources: Claude Code docs: best practices

Make the check a gate: the Stop hook

Instructions in CLAUDE.md are advisory. The docs say that, unlike CLAUDE.md instructions, hooks are deterministic and guarantee the action happens. A Stop hook runs when the main agent finishes responding, and it can refuse to let the turn end. Per the hooks reference, a hook that exits with code 2 prevents Claude from stopping and continues the conversation, and the stderr message goes to Claude as the explanation. The alternative is JSON output with decision set to "block" and a reason, which the reference says is required when blocking.

Sources: Claude Code docs: hooks reference

The practical recipe is short. The hook script runs your test command, lint, or type check. If it fails, the script must itself exit 2 and print the failing tail to stderr. Passing the test runner's own exit 1 straight through does not block: the reference treats any exit code other than 0 and 2 as a non-blocking error. Claude gets the failure as feedback and keeps working. If it passes, the script exits 0 and the turn ends. Now the verdict comes from the exit code of a command you chose, not from the agent's summary. One limit from the reference: Stop hooks do not run when the turn ends from a user interrupt, and API errors fire a different event, StopFailure, instead.

Two details from the docs matter. First, loop protection. Stop hooks receive a stop_hook_active field that is true when Claude Code is already continuing because of a stop hook. The reference says to check this value or process the transcript "to avoid blocking on a condition that will never resolve". Claude Code also applies a cap: after stop hooks have continued the turn eight times in a row, it overrides the next block and ends the turn. The count resets each time Claude calls a tool, and the cap can be raised with the CLAUDE_CODE_STOP_HOOK_BLOCK_CAP environment variable. A hook that fails forever will therefore not trap the session, but it will not hold forever either, so a final message that follows a capped stop deserves extra suspicion.

Sources: Claude Code docs: hooks reference, Stop input

Second, the hook receives last_assistant_message, the text of the agent's final response. A hook can use it, for example to refuse a turn that claims success without a command output in it. A command-based check, though, is stronger than a text check, because a text check can still be satisfied by a convincing paragraph.

Sources: Claude Code docs: hooks reference, Stop input

The docs list other ways to harden the stop. The /goal command sets a condition that a separate evaluator re-checks after every turn, described as a built-in shortcut for a session-scoped prompt-based Stop hook. Prompt and agent hook types let a model or a subagent evaluate conditions before returning a decision, though the docs mark agent hooks as experimental.

Sources: Claude Code docs: best practices, Claude Code docs: hooks reference

A second gate before the commit

Stopping is one boundary. Committing is another, and a stronger one, because it is where unverified work enters shared history. A PreToolUse hook can block a tool call before it executes, so you can refuse a git commit Bash call unless a check passed. Such a hook reads the command text, so an unusual spelling of the command can slip past it. A plain git pre-commit hook does the same job outside Claude Code and also guards your own commits. Pair the two: the Stop hook catches false completion in the session, the commit gate catches what slipped past it.

Sources: Claude Code docs: hooks reference

Insight

A hook proves only what its command checks. If the test suite is thin, a green hook is a thin guarantee. The gate removes the agent's self-report from the loop. It does not make the check itself good.

A short routine you can adopt today

  • Write the pass/fail check first, and state in the prompt what "done" means.
  • Require the command and its raw output in the final message. No output, no acceptance.
  • Read the exit code, not the summary line. Look for separate error lines around the totals.
  • Run the real feature once: the page, the endpoint, the job. Do not stop at unit tests.
  • Add a Stop hook that runs the check and exits 2 on failure, and handle stop_hook_active.
  • Before merge, have a fresh-context reviewer compare the diff with the written requirements.
  • If the agent reports success on the same bug twice and it recurs, stop retrying in the same session. See Why Claude Code Keeps Fixing the Same Bug.

Where TLM Forge fits

TLM Forge adds independent checks alongside the self-report. On Medium and Deep tasks, a final review wave of separate reviewer agents examines the work, and the phase-auditor checks that what the spec promised was delivered and that out-of-scope items were respected. A commit hook blocks git commit, git push and gh pr merge when the code changed since the final review, and Deep work is verified on a real environment.

The limits are real. The reviewers are LLM agents and can miss things. Light tasks get no review agents at all. The commit hook matches the command text of those three commands and turns off with TLMFORGE_HOOKS=0. None of this proves the code is correct; it adds independent looks before work ships, and you still read the evidence. See also the final audit page.

Frequently asked questions

01Why does Claude Code say tests pass when they do not?

Common patterns are that it ran only part of the suite, ran a stale build, or inferred a result from reading code without running anything. The summary line can also hide a separate error. Ask for the exact command and raw output, and treat the exit code as the verdict.

02How do I make Claude Code prove it ran the tests?

Require the command and its full output in the same message as the claim, and add a Stop hook that runs the tests itself and blocks the turn from ending when they fail. Then the result comes from a script, not from the agent.

03Can a Stop hook force Claude Code to keep working?

Yes. A Stop hook can block stopping by exiting with code 2 or returning a block decision with a reason. Claude Code caps consecutive blocks at eight by default, and hooks should check the stop_hook_active field to avoid unresolvable loops.

04Is Claude lying when it claims a fix that does not work?

The public reports describe false claims but cannot show intent or cause. A status message is generated text, and it can be wrong. Whatever the cause, the practical answer is the same: do not accept a claim without evidence you can check independently.

05What should I check besides the pass count?

Check the exit code, the number of tests collected, import or fixture errors printed apart from the totals, and the diff for unexpected files. For user-facing changes, run the real feature once and look at the actual output.

Ship AI-written code you can trust

TLM Forge is the missing process layer for Claude Code: a spec audit, independent multi-agent review, enforced TDD, and an adversarial red-team gate.

Get TLM Forge