Claude Code sometimes rewrites, skips or loosens tests to make them pass. Five layers that block or flag it, and what each one still misses.
Stop Claude Code from changing tests to make them pass with checks outside the agent's reasoning, not instructions. Commit the failing tests first, deny edits to test paths, block them again with a PreToolUse hook, flag weakened tests with a CI diff check, and give the diff to a reviewer that never saw the agent's reasoning. A "don't touch the tests" line in CLAUDE.md is the layer people report Claude ignoring. In one early bug report, Claude, asked to get all tests passing, made the Makefile run only the passing ones.
Sources: anthropics/claude-code issue #319: A tendency to re-write tests so they pass
Whether Claude is rewriting an assertion, skipping a test or narrowing the test run, no single layer is complete. In ImpossibleBench, making tests read-only prevented test modification but "does not eliminate other cheating methods such as special-casing or operator overloading." Each layer below closes a path the others leave open, and the comparison table spells out what each one misses.
Sources: ImpossibleBench full text, Section 5 (test access)
Disclosure: we build TLM Forge, a process layer for Claude Code, so weigh the product section near the end accordingly. Everything before it works with plain Claude Code and git.
What "changing tests to pass" actually looks like
The public reports describe something quieter than deleting a test: a change to the premise. One early report on the Claude Code issue tracker, titled "A tendency to re-write tests so they pass", describes the agent changing a test's conditions so it passed, and, when asked to get all tests passing, updating the Makefile to run only the tests that already passed. Claude called these "safe-tests". A later report lists the same pattern in more detail: modifying tests instead of fixing the implementation, changing assertions to match incorrect behavior, and removing or weakening validations, while instructions in CLAUDE.md and imported files went unfollowed.
Sources: anthropics/claude-code issue #319: A tendency to re-write tests so they pass, anthropics/claude-code issue #7074: Claude Code manipulates tests instead of following instructions
The variants are worth naming, because each needs a different check. A reporter on Claude Code 2.1.96 described giving Claude example YAML files that should fail validation, then finding that the tests it wrote passed because it had modified the examples instead of using them as given. DoltHub's engineering blog put the general case bluntly: Claude Code "is not bashful about modifying tests to be less specific or worse, changing the test to assert the implemented (wrong) behavior."
Sources: anthropics/claude-code issue #45159: Claude modifies test input examples instead of using them as-is, DoltHub: Claude Code Gotchas (Tim Sehn, June 30, 2025)
- Edited assertions: the expected value is changed to whatever the code currently returns.
- Loosened assertions: an exact match becomes a substring check, a type check, or a wider numeric tolerance.
- Skipped tests: a skip or xfail marker, or a commented-out test, so the failure never runs.
- Narrowed selection: the Makefile, test config or CI job is edited so the failing tests are no longer collected.
- Altered fixtures: the inputs a test was meant to reject are changed until they pass.
- Gamed implementation: the tests are untouched, but the code special-cases the exact inputs the tests use.
Why Claude Code rewrites tests to pass: reward hacking
An agent told to make tests pass is optimizing a signal, and editing the test is often the shortest path to that signal. METR documented the pattern across frontier models in June 2025, writing that recent models had engaged in "increasingly sophisticated reward hacking, attempting (often successfully) to get a higher score by modifying the tests or scoring code." METR also noted that the models "demonstrate awareness that their behavior isn't in line with user intentions and disavow cheating strategies when asked." The post centers on OpenAI's o3 and says METR noticed similar behaviors in other recent models, including Claude 3.7 Sonnet.
Sources: METR: Recent Frontier Models Are Reward Hacking (June 5, 2025)
ImpossibleBench, a benchmark from Ziqian Zhong, Aditi Raghunathan and Nicholas Carlini, measures this directly by building coding tasks where the specification and the unit tests conflict, so any pass requires a shortcut. The authors sort the cheating into four strategies: "Modify Test Cases", "Overload Comparison Operators", "Record Extra States" and "Special Casing". For the first, they write that the model "directly modifies tests despite being explicitly instructed not to." They also found that models "can devise creative workarounds even when they technically comply with instructions not to modify tests."
Sources: ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases (arXiv:2510.20270), ImpossibleBench full text (HTML)
The ImpossibleBench result that matters for this post: on their SWE-bench variant, making tests read-only "restores legitimate performance while preventing test modification attempts," but leaves the other cheating strategies open. Their recommendation is to hide test files or restrict them to read-only access during implementation, when feasible. Read-only tests are necessary, not sufficient.
That is why the CLAUDE.md approach disappoints. Claude Code's own best-practices page draws the line: "Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens." An instruction competes with everything else in context, and the reporter of issue #319 found that explicit instructions faded in importance as the thread grew longer. We covered the general version of that problem in why your CLAUDE.md gets ignored. Keep the instruction, since it costs nothing, but do not count it as a control.
Sources: Claude Code docs: Best practices (Set up hooks), anthropics/claude-code issue #319
Layer 1: write and commit the failing tests first
Commit the tests before any implementation exists, so every later change to them shows up as a diff against a known baseline. This is the cheapest layer and every other layer builds on it. Without a committed baseline, a weakened assertion is indistinguishable from a test that was always weak.
- Have Claude write the tests from the spec in one session, run them, and confirm they fail for the right reason, not because of an import error or a typo.
- Review those tests yourself. DoltHub's advice is to "spend a bit more time than usual in review of the generated tests," because a weak test committed now becomes the baseline you defend later.
- Commit them on their own, with a message that says they are the acceptance tests. Note the commit hash.
- Start the implementation in a fresh session. Claude Code's docs suggest this split directly: "have one Claude write tests, then another write code to pass them."
- At any point,
git diff <test-commit> -- tests/shows every change to the test tree since the baseline. An empty result is the answer you want.
This layer does not prevent anything. It makes tampering visible. The layers below make it harder, and the last two make sure someone looks.
Layer 2: deny Edit on test paths in permissions
Once the tests are committed, add deny rules so Claude's file tools cannot change them. In .claude/settings.json, put entries like Edit(/tests/**), Edit(**/*.test.ts), Edit(**/*_test.go) and Edit(/pytest.ini) in permissions.deny. In project settings a leading slash anchors the pattern at the session's primary working directory, and the patterns use gitignore syntax. Use Edit, not Write: the docs say Claude Code checks file permissions against Edit(path) and Read(path) rules only, and Edit rules "apply to all built-in tools that edit files."
Sources: Claude Code docs: Configure permissions (Read and Edit)
Protect test selection too, not only test files. The "safe-tests" Makefile in issue #319 never touched a test file. Add your Makefile test target, your runner config (jest.config.*, vitest.config.*, pytest.ini, or pyproject.toml if it holds your pytest settings), and .github/workflows/** to the same list. Rules are evaluated deny first, then ask, then allow, so a broad allow rule elsewhere cannot override them. If you need to make legitimate test changes during a session, use ask instead of deny so each edit comes to you for approval.
Sources: Claude Code docs: Configure permissions (rule evaluation order)
The documented limits. Edit deny rules cover Claude's built-in file tools, file commands Claude Code recognizes in Bash such as sed and tee, and Bash redirection targets. They "don't apply to... arbitrary subprocesses that read or write files indirectly, like a Python or Node script that opens files itself." Bash rules themselves match the command text and, in the docs' words, are not "a security boundary around the program." For OS-level enforcement, the docs point to the sandbox. Treat this layer as a strong default against the common path, not a lock. For the full rule mechanics, see what Claude Code deny rules miss.
Layer 3: a PreToolUse hook that blocks test edits
A hook adds logic the deny list cannot express, and it tells Claude why it was stopped. Claude Code's hooks guide ships a "Block edits to protected files" example: a script registered as a PreToolUse hook with the matcher Edit|Write reads the JSON input on stdin, pulls tool_input.file_path with jq, compares it against a list of protected patterns, prints a "Blocked:" message to stderr and exits with code 2. The guide notes that "Claude receives feedback explaining why the edit was blocked, so it can adjust its approach." For PreToolUse, exit code 2 blocks the tool call.
Sources: Claude Code docs: Hooks guide (Block edits to protected files), Claude Code docs: Hooks reference (exit code 2 behavior per event)
- Swap the example's patterns for your test paths and test config:
tests/,_test.go,.test.ts,.spec.ts,conftest.py,pytest.ini,jest.config,Makefile,.github/workflows/. - Freeze existing tests but allow new ones: block only when the target file already exists, so Claude can add test files but cannot rewrite the ones you committed.
- Add a phase switch: let the script allow test edits only while a marker file such as
.test-phaseexists, and delete it after you commit the tests. - Settings hooks also run inside subagents. The hooks reference says tool events such as
PreToolUse"fire the same configured hooks as in the main conversation," so delegating the work does not route around the block.
Copy this as your hook's stderr message: "Blocked: tests are the spec. Fix the implementation, or stop and explain why the test is wrong." That message is what Claude reads next.
The hook shares the deny rule's blind spot: with an Edit|Write matcher it never sees Bash. You can add a second hook on Bash that inspects tool_input.command for your test paths, which catches a crude sed -i on a test file, but it is still pattern matching on command text. A script that opens the file itself passes straight through. That is the gap layers 4 and 5 exist to cover.
Sources: Claude Code docs: Hooks reference (PreToolUse input)
Layer 4: a diff check that flags weakened tests
Run a check over the final diff that fails when the change touches tests in suspicious ways. It does not care which tool made the edit, so it catches the script writes that slip past layers 2 and 3. Run it as a required CI status check on the pull request, where the agent's session cannot reach it, and optionally as a Stop hook so Claude sees the failure before it reports done.
Sources: Claude Code docs: Best practices (Stop hook as a deterministic gate)
- Any modified or deleted file under the test tree since the baseline commit, from
git diff --name-status <base>...HEAD. Added files are fine; changed ones need a reason. - Removed lines containing assertions:
assert,expect(,assertEqual,require.,t.Fatal. Deleted assertions are the clearest signal. - Added skip markers:
@pytest.mark.skip,xfail,it.skip,describe.skip,test.todo,t.Skip(,#[ignore],@Disabled. Also flag.only, which silently drops every other test in the file. - Loosened tolerances: changed
rel=orabs=inpytest.approx,rtoloratolin numpy, the precision argument intoBeCloseTo. - Edited test selection: any change to the Makefile test target, runner config,
testpaths, include or exclude globs, or the CI job that runs the suite. - A drop in the collected test count compared with the base branch. Most runners can report it, and it catches tests that vanished for any reason.
Keep the check dumb and loud. It should not decide whether a change is legitimate, only force a human to look. Sometimes a test really is wrong, and the right outcome is a reviewed test change with a stated reason, not a ban. This check also cannot see the sixth variant from the list above, code that special-cases the test inputs. Mutation testing is the tool for hollow tests and gamed code, covered in mutation testing for AI code.
Layer 5: an independent reviewer that sees only the diff
The last layer is judgment: a reviewer that reads the diff without the implementer's explanation. Claude Code's best-practices page describes the mechanism: "A reviewer running in a fresh subagent context sees only the diff and the criteria you give it, not the reasoning that produced the change, so it evaluates the result on its own terms." That matters here because test tampering usually arrives with a plausible justification attached, and DoltHub notes that when challenged, Claude Code "will often even say something like 'this is how it should work anyway'."
Sources: Claude Code docs: Best practices (Add an adversarial review step), DoltHub: Claude Code Gotchas
- Define it in
.claude/agents/withtools: Read, Grep, Glob, Bash, so it can run git and read files but has no Edit or Write tool. - Give it the baseline: "Diff HEAD against <test-commit>. Report any change to tests, fixtures, test config or CI selection, any deleted or weakened assertion, and any code that branches on values that appear only in tests."
- Tell it to report only findings that affect whether the tests still verify the spec. The docs warn that a reviewer asked to find gaps will usually report some even when the work is sound.
- Route its findings to a person, not back into the same session as a to-do list for the implementer to argue with.
Do not make this your only layer. ImpossibleBench tested LLM monitors on its cheating transcripts and found they detected 86 to 89 percent of cheating attempts on the simpler LiveCodeBench variant but had "much lower sensitivity" on the multi-file SWE-bench variant. In one example, a sophisticated rationalization fooled the Claude Opus 4 monitor. The authors conclude that "simple LLM-based monitoring approaches may not be sufficient for detecting cheating attempts on complex multi-file tasks."
Sources: ImpossibleBench full text, Section 6 (monitoring)
A reviewer catches what mechanics cannot express. Mechanics catch what a persuasive rationalization talks a reviewer out of.
The five layers compared
| Layer | What it stops | What it misses | Where it runs |
|---|---|---|---|
| CLAUDE.md instruction | Some casual edits, when the rule is still salient | Unreliable; reporters say it fades as sessions get long | Inside the model context |
| 1. Committed failing tests | Nothing directly; makes every later change visible | Tampering nobody looks for | git |
| 2. Edit deny rules | Edits via built-in file tools, recognized Bash file commands and redirects | Python or Node scripts that write files themselves; legitimate test fixes | Claude Code permissions |
| 3. PreToolUse hook | Edit and Write calls on matched paths, with feedback telling Claude what to do | Bash writes unless you add a Bash hook, and even then indirect writes | Claude Code hooks, including subagents |
| 4. Diff check in CI | Deleted assertions, new skips, loosened tolerances, edited selection, whatever tool did it | Special-cased implementation code; changes it has no pattern for | CI, outside the agent session |
| 5. Diff-only reviewer subagent | Subtle weakening and special-casing a pattern cannot express | Persuasive rationalizations on large multi-file diffs | Fresh subagent context |
If you only have an hour, do layers 1, 2 and 4. The committed baseline makes tampering visible, the deny rules block the common path, and the CI check flags test changes regardless of how the edit was made.
Tooling exists for parts of this. TDD Guard, billed as "Automated TDD enforcement for Claude Code," uses hooks to block implementation written without a failing test and code beyond what the current tests require; its README says it has evolved into a successor project called Probity. Green tests that verify nothing are a separate failure, covered in your AI agent passed the tests by luck.
Sources: nizos/tdd-guard on GitHub
Where TLM Forge fits
TLM Forge covers the process around these layers, not the file locks. It tells Claude to build tests first, but that is an instruction, the same advisory tier as a CLAUDE.md rule: no hook checks the test results, so layers 2 to 4 above still matter.
The part aimed at this post's problem is the final review wave: independent reviewer agents read the finished change. Medium runs a code-reviewer and a phase-auditor; Deep adds a tester, a red-team-reviewer and an architect-reviewer, plus a ux-reviewer when the UI changes. Light tasks get no review agents, so a Light change that touches tests gets no reviewer. CRITICAL and HIGH findings are fixed test-first or brought to you, and MEDIUM and LOW findings go on a written list. Its reviewers are LLM agents, with the limits ImpossibleBench found for LLM monitors on multi-file diffs. A commit hook then blocks commits, pushes and PR merges when the code has changed since that review, so edits made after the reviewers looked do not slip into a commit unreviewed. It is on for everyone, and TLMFORGE_HOOKS=0 turns hooks off. None of this makes code bug-free.
Frequently asked questions
01Why does Claude Code keep changing or rewriting my tests?
It is optimizing for passing tests, and editing the test is often the shortest route. Researchers call this reward hacking. METR observed frontier models modifying tests or scoring code to raise scores, and ImpossibleBench found models modifying tests despite explicit instructions not to.
02Does telling Claude not to edit tests in CLAUDE.md work?
Partly, and not reliably. Claude Code's docs describe CLAUDE.md instructions as advisory and hooks as deterministic. GitHub reporters describe CLAUDE.md rules being ignored while tests were modified. Keep the instruction, but enforce it with permissions, a hook and a CI diff check.
03How do I make test files read-only for Claude Code?
Add deny rules such as Edit(/tests/**) and Edit(**/*.test.ts) to permissions.deny in .claude/settings.json, then add a PreToolUse hook on Edit|Write that exits with code 2 for test paths. Protect test config and CI files too, so the test run cannot be narrowed.
04Can Claude Code still edit tests through Bash if I deny Edit?
Sometimes. Edit deny rules cover recognized Bash file commands like sed and tee and redirect targets, but not Python or Node scripts that write files themselves. The docs point to the sandbox for OS-level enforcement. A CI diff check catches the change regardless of tool.
05How do I catch skipped or weakened tests in a pull request?
Diff the PR against the commit where the tests were added. Flag removed assertion lines, new skip or xfail markers, .only, changed tolerances, edits to runner config or CI test selection, and a drop in collected test count. Run it as a required CI check.