Claude Code fixed the bug and broke a caller? How to map the blast radius, run the right tests and compare behavior before and after the change.
When Claude Code fixes one bug and something nearby breaks, the story in several user reports is that the session did not check, or checked too shallowly, what else depended on the code it changed. The fix is to make that check a step: read every caller before the edit, and judge the result against a baseline run rather than against tests written alongside the change.
Disclosure: we build TLM Forge, a process layer for Claude Code, so weigh the product section accordingly. Most of this post applies with or without us.
What a fix that breaks something else looks like
This is a different problem from Why Claude Code Keeps Fixing the Same Bug, where one bug survives repeated patches. Here the original bug is gone and something new fails: a caller that relied on the old return value, a report that read a column the fix renamed, a config key nobody remembered. It also differs from Why Claude Code Edits Files You Never Named, where the damage comes from files outside the task. In this case every edit is in scope, and the breakage can land on code the agent never opened.
Developers describe the pattern in their own words. One Hacker News post about AI coding assistants in general lists "Zero holistic awareness" among its gripes and says the assistant "couldn't care less if its changes break other parts of the system." A user-filed GitHub issue against Claude Code says it "repeatedly makes changes to files without first understanding what depends on them," and its examples include an apply script and a kustomize file, not only application code. Another issue describes a fix for a deleted object that reappeared after a restart. Before the change, reading an old version after a restart returned a clean "no such version" error. After it, the same read returned a different version's bytes under the old version's checksum, and the regression test the model wrote checked the version list and the delete marker but never that read. These are individual reports, not measurements, and none of them says how often this happens.
Sources: Hacker News post by sollawen, anthropics/claude-code issue 60038, anthropics/claude-code issue 96463
Four kinds of nearby breakage and what catches each
A type checker catches a changed signature in a typed language and says nothing about a changed rounding rule. Hyrum's Law explains why the second kind matters: with enough users of an API, in the law's words, "all observable behaviors of your system will be depended on by somebody." The table pairs each kind of breakage with the check that finds it. The examples are illustrative.
Sources: Hyrum's Law
| Kind of breakage | Illustrative example | What catches it | What misses it |
|---|---|---|---|
| Contract change | A renamed field, changed return type or new required argument | Compiler or type checker, language server diagnostics, tests of the callers | Callers reached by string name, reflection or untyped code |
| Behavior shift | Same signature, but rounding, ordering or a default changed for some inputs | Characterization tests, a before and after comparison on real inputs | Type checks, and any test written from the new behavior |
| Newly reachable path | The fix makes an old fallback branch run, and it returns wrong data instead of an error | Tests that exercise every path whose inputs the change alters | A regression test that covers only the fixed path |
| Out-of-code coupling | A config key, SQL view, template, script or doc that depends on the old name or behavior | Text search for the key, change history, running the integrated system | Reference lookups and unit tests that never load the config |
What the research says about agents and regressions
SWE-bench, a benchmark built from real GitHub issues, puts the regression check into its scoring. Each task carries two lists of tests, named FAIL_TO_PASS and PASS_TO_PASS in the dataset: tests that fail before the fix and should pass after, and tests that should pass both before and after. A task counts as solved only if all of them pass, and the paper says the second group is included "to ensure that the revision does not break or violate any existing expected behavior". By that standard, a fix that makes a previously passing test fail is a failure, though only a test that exists can catch it.
Sources: SWE-bench paper (arXiv 2310.06770), SWE-bench dataset card
A second benchmark, SWE-CI, measures regressions over time. It has 100 tasks built from Python repositories, each spanning on average 233 days and 71 commits of real history, and it defines a regression as a code change that makes a previously passing test fail. Across 20 models from 8 providers, the authors report that most models had a zero-regression rate below 0.25, meaning fewer than a quarter of samples got through the whole maintenance run without breaking a passing test, and that only the two Claude Opus models exceeded 0.5. Two cautions apply. Models ran in an architect and programmer loop defined by the benchmark, with iFlow CLI as the default harness. The paper does not mention Claude Code, so the numbers do not describe a Claude Code session. The benchmark can also only see regressions that the repository tests can see.
Sources: SWE-CI paper (arXiv 2603.03823)
An arXiv preprint called TDAD examined the agent side. On SWE-bench Verified with a small open-weight model (Qwen3-Coder 30B, 4-bit, run locally on 100 instances), the baseline agent caused 562 pass-to-pass test failures, a test-level regression rate of 6.08%. Adding a map from source files to the tests that exercise them, plus a short skill telling the agent to run the related tests, on top of TDD-style instructions cut that to 155 failures, or 1.82%. The TDD-style instructions alone, with no map, raised it to 9.94%. The authors conclude that, for smaller models, telling an agent which tests to check beats prescribing a workflow. This is one preprint, small open-weight Qwen models and one benchmark, and it did not test Claude models. The map-equipped agent also returned empty patches more often (26% against 14% for the baseline), so its figures cover fewer patches (74 non-empty ones against 86), and its resolution rate dipped from 31% to 29%. Counted per patch instead of per test, the share of non-empty patches with at least one previously passing test failing was 30.2% for the baseline and 33.3% with the map, a gap the authors attribute to the lower generation rate of the map-equipped agent. Read it as a pointer to what to test, not as an effect size for your setup.
Sources: TDAD paper (arXiv 2603.17973)
Anthropic's write-up on CI makes a similar point. In its account of a test-selection service, the post's author notes that humans are good at judging which test failures do not apply to them while agents need more context and direction, and adds: "When they get a specific set of valid tests, they can self-verify and iterate more effectively." The longer story is in How Anthropic Fixed CI After Claude Wrote 80% of Its Code.
Sources: Anthropic: Agentic coding is straining CI
Step 1: list the callers, then make the agent read each one
Start with everything the fix will change: function names, exported types, database columns, config keys and error messages that callers might match on. Claude Code finds callers with text search: on macOS, Linux and WSL it runs grep through the Bash tool by default, and on Windows the dedicated Grep tool is in the default set. For typed languages, the docs describe a code intelligence plugin that activates an LSP tool able to find all references to a symbol and trace call hierarchies, and the same setup reports type errors after each file edit. The tool stays inactive until you install a plugin for your language, and cloud sessions do not start plugin language servers.
Sources: Claude Code docs: Tools reference (Grep and LSP), Claude Code docs: Code intelligence plugins
Both kinds of search have blind spots. A handler looked up by name from a string, a column named in a SQL view and a key read from a YAML file never appear as references to the symbol. Finding a caller is also not the same as checking it. One GitHub issue describes a model asked to review a pull request that ran a caller grep, got six call sites and two test patches, and gave a verdict without opening any of them. That is a single report, but it names the gap: a list of callers helps only if each one is read against the new behavior.
Sources: anthropics/claude-code issue 107592
The best-practices page suggests raising effort with /effort high or /effort xhigh when getting it right depends on cases that are quick to overlook, and its examples include a schema change where Claude has to find every reader of the old shape. A change to a shared helper is the same kind of task. Ask for the list before the edit.
Sources: Claude Code docs: Best practices
Prompt to paste before the fix: "Before you edit anything, list every symbol this fix will change: names, signatures, return values, error behavior, config keys and columns. For each one, find every caller and every non-code reference (config, SQL, templates, scripts, docs) and open each one. Give me a table with the location, what it assumes about the old behavior, and whether the fix keeps that assumption true. Do not edit until I have seen the table."
Step 2: use history to find coupling that search cannot see
Version history records what changed together. A 2004 ICSE paper mined it for rules of the form "Programmers who changed these functions also changed...", and reported that such rules can show coupling that program analysis cannot detect. After an initial change, its prototype could correctly predict 26% of the further files to be changed, so history is a hint and never a complete list. A rough version for your own repo counts which files most often changed in the same commits as the file you are about to edit. Run it from the repository root and replace FILE with the file's path in both places: git log --follow --no-merges -n 50 --format=%H -- FILE | while read -r c; do git show --name-only --format= "$c"; done | grep -v -x -F FILE | sort | uniq -c | sort -rn | head. In a toy repository built so that a YAML file and a handler reached by string name never mention the function (git 2.54.0), it listed both, and a text search for the function name found neither. A repo-wide reformat adds noise, so ignore commits that touch dozens of files, and after a rename the file's old path shows up in the list too.
Sources: Zimmermann et al., Mining Version Histories to Guide Software Changes (ICSE 2004), git-log documentation, git-show documentation
Step 3: run a baseline, then the tests that depend on the change
The TDAD authors describe the choice an agent faces on a large codebase: run every test, which is too slow, or run only the tests near the changed files, which misses indirect dependencies. A baseline and a deliberate test selection address both.
Sources: TDAD paper (arXiv 2603.17973)
- Record a baseline first.
git worktree add --detach ../base HEADchecks out the committed state in a separate folder, so you can run the suite without your edits. If the agent has already committed, use the commit you started from instead of HEAD. A worktree holds only committed files, so install dependencies and copy untracked config such as .env before the run, and save the names of the failing tests. Remove it afterwards withgit worktree remove --force ../base. - After the change, run the full suite and compare failing test names with the baseline. Tests that passed before and fail now are the regression candidates: rerun each one to rule out a flaky test. Tests that failed in both runs were already failing.
- If the full suite is too slow, select tests by what reaches the changed code, directly or through other modules, and name them in the prompt. The TDAD preprint and Anthropic's CI write-up both point at giving the agent a specific set of tests.
- Make the check something Claude Code runs rather than something it remembers. The best-practices page says CLAUDE.md is for guidelines and Claude is allowed to decide a guideline does not apply, and it shows a Stop hook for checks that cannot be optional. Adapting its typecheck example to a test command gives
{"hooks": {"Stop": [{"hooks": [{"type": "command", "command": "npm test 1>&2 || exit 2"}]}]}}in .claude/settings.json, with your own test command in place of npm test. Exit code 2 blocks the stop and sends the stderr output back to Claude. Stop hooks fire whenever Claude finishes responding, not only at task completion, so this one also runs on turns that only answer a question, and Claude Code overrides it after eight blocks in a row with no tool call between them, so keep the command fast. It also blocks on any failing test, including ones that were already failing, so it suits a suite that is green at baseline.
Step 4: pin behavior first, and distrust tests written with the change
A regression test the agent writes alongside the fix can cover only the path it fixed. In the deleted-object issue above, the test checked the version list and the delete marker, and the changed read path went untested. A test written from the new behavior shares the assumptions of the change.
Sources: anthropics/claude-code issue 96463
Characterization tests are built for this gap. Michael Feathers describes the process as writing a test with a dummy expected value, running it, and reading the actual result off the failure, and he writes: "The purpose of characterization testing is to document your system’s actual behavior, not check for the behavior you wish your system had." He adds: "When a system goes into production, in a way, it becomes its own specification." A Stack Overflow Blog article on the same technique notes that when an assistant drafts a big refactor and updates the tests in the same pass, a baseline captured beforehand is one of the very few checks that does not share the change's assumptions.
Sources: Michael Feathers: Characterization Testing, Stack Overflow Blog: The Regression Tests Nobody Writes
In practice, before the fix, record what the code does for the inputs real callers send and commit that as a test. When it fails after the change, ask the question the Stack Overflow article poses: "did we mean that?" Sometimes the answer is yes and the new behavior becomes the baseline. Sometimes it is a regression that no intent-based test would have caught. How to Refactor With AI Without Breaking Things covers characterization tests and step size for refactors in more detail.
Sources: Stack Overflow Blog: The Regression Tests Nobody Writes
When a lighter touch is enough
None of this belongs on every change. A rename inside one private function, a new file nothing imports or a copy edit has no callers to read, and the baseline plus the normal test run is enough. The caller table earns its cost on shared helpers, public interfaces, database columns, config keys and anything many files import. The reading costs tokens and time, and the best-practices page warns, about review prompts, that a reviewer asked to find problems will always find something, so keep the prompt on callers and behavior, not style.
Sources: Claude Code docs: Best practices
Where TLM Forge fits
TLM Forge sizes each task Light, Medium or Deep. Light gets no review agents, so a change sized Light that still breaks a caller has no independent reviewer looking for it. Medium and Deep show a design for your OK before building, a cheap moment to ask which callers the change touches. The final review wave adds independent reviewer agents: Medium runs a code-reviewer and a phase-auditor, and Deep adds a tester, a red-team-reviewer and an architect-reviewer.
The limit: those reviewers are LLM agents and can miss a caller. TLM Forge tells Claude to build tests first, but that is an instruction and no hook checks test results, so your own suite and the baseline comparison above stay the safety net.
Frequently asked questions
01Why does Claude Code break existing features when it fixes a bug?
When it happens, the fix changed behavior that other code relied on, and no check in the session covered those places. A test covering only the fixed path can pass while a caller, config file or nearby path quietly fails. Individual user reports describe this, but they do not show how often it happens.
02How do I stop Claude Code from breaking existing code?
Make the check mandatory and specific. Have it list and read every caller before editing, run a baseline suite before and after, and add a Stop hook that runs your tests. CLAUDE.md rules help, but the docs call them guidelines that Claude may decide do not apply. This lowers the risk without removing it.
03Does Claude Code check callers before changing a function?
It can search for callers, and a language server plugin adds find-references for typed languages, but a search tool does not make the agent open and check each result. Ask for the list before the edit and require each caller to be read.
04What tests should I run after an AI code change?
Run a baseline before the change, then the full suite after, or at least every test that reaches the changed code, directly or through other modules, and not only the file beside it. Compare failing test names with the baseline. Add characterization tests for behavior callers depend on that no test covers.
05What is a characterization test?
A test that records what code does for given inputs, whether or not that behavior is correct. It works as a safety net for change: if a later edit alters the result, the test fails and you decide whether the change was intended.