← Back to BlogSpec-Driven Dev

Claude Code Skipped Requirements? How to Audit It

Claude Code can pass its tests and still skip requirements. How to audit what the spec promised against what shipped, with prompts to copy.

If Claude Code built most of what you asked and quietly left the rest out, the fix is an audit that runs requirement by requirement. Number the requirements before the build, then make every one point to the code that implements it and a test that would fail without it. A passing test suite and a reasonable-looking diff cannot, on their own, show you what is missing.

This post covers what omission looks like, what research and Anthropic's docs say about it, a ledger and checks you can copy, and where they stop working. It is about work that is absent. An agent claiming success without evidence is a different problem, covered in Claude Code Said It's Done. It Wasn't., and changes outside the task belong to Why Claude Code Edits Files You Never Named.

Disclosure: we build TLM Forge, a process layer for Claude Code, so weigh the product section accordingly. Almost everything below works with plain Claude Code.

What an omission looks like

One user's issue on the Claude Code tracker describes a long session of multi-part messages in which Claude answered some parts and silently dropped others. The replies read as complete and never said an item was deferred, declined or pending. The reporter separated this from scope drift: scope drift adds or substitutes work outside the mandate, while this drops work inside it. That is one user's account of one session, not a rate.

Sources: anthropics/claude-code issue 85255

A second user, reviewing pull requests with Claude, reports that Claude rejected two as "already covered by our branch" after implementing only part of each and never listing the rest, then called the related issues ready to close. The user closed them and shipped before finding the gaps. A third report, from a Python codebase, describes functions defined but never called from another module and settings that no executing code read. That reporter's check was a grep for the definition plus a grep for the call site. All three are single-user accounts, so read them as symptom descriptions.

Sources: anthropics/claude-code issue 100341, anthropics/claude-code issue 49674

Anthropic's best-practices page frames the problem plainly: "Claude stops when the work looks done. If it has nothing to run that tells it whether the work actually is done, 'looks done' is all it has to go on." The same page says the /goal command matters in long background runs, "where Claude sometimes ends a turn with a progress summary while items are still open." Completeness is something to verify, not assume.

Sources: Claude Code docs: Best practices

What the research says, and what it cannot say

A University of Montana paper on arXiv measured requirement coverage directly. The authors took 25 GitHub projects whose READMEs had numbered feature lists, gave the requirements to GPT-5, and checked the code against those lists. Direct prompting ("Generate complete code for this project") completed 80.5% of tasks on average. A staged workflow (requirements analysis, design, test specification, then code) reached 96.9%, yet 8 of the 25 projects stayed incomplete, with 19 missing tasks between them. Examples include a missing admin profile update endpoint and an incomplete RabbitMQ message queue integration. The authors report that the models failed less on syntax or type correctness, and that the remaining errors clustered around high-level functional omissions.

Sources: arXiv 2602.02896: Failure-Aware Enhancements for LLM Code Generation

Two details matter for your own process. First, the paper's baseline table lists the original human-written projects at 95.7% average completion, so by the authors' measure they did not fully cover their feature lists either. Second, the authors asked the same model to critique its own output: "Review the generated code carefully. Is it complete? What requirements are missing or only partially implemented?" It reached full coverage on two of the six challenge projects, partly helped a third and made no progress on three. It caught a checkout endpoint that never compared the requested quantity with stock, a gap visible in the code. It could not identify the missing message queue configuration, which is not inferable from application code alone. The authors call self-critique most effective on failures that are "code-reviewable" without external knowledge.

Sources: arXiv 2602.02896 (self-critique results and discussion)

A design step did not guarantee coverage either. In the pipeline where GPT-5 wrote the design and another model wrote the code, the shopping cart design covered the basic flow but omitted validation rules, and the inventory check was still missing from the result. Check a design against your requirement list before building.

Sources: arXiv 2602.02896 (project P10 discussion)

The study has limits. It generated whole projects from README feature lists in prompt workflows, not agentic edits to an existing repository in Claude Code. Completion was judged by manual code inspection without running the projects, only six projects went into the enhancement tests, and it is a single study. Read it as evidence of where omissions cluster, not as a rate for any tool.

Sources: arXiv 2602.02896 (threats to validity)

A separate paper in the Findings of EMNLP 2025 measures one way this can happen. It tested ten LLMs, including GPT-4o, Claude 3.5 Sonnet and Llama 3.1 8B, on Python problems that each carried one to six extra style instructions, such as a license notice, docstrings or a line-length limit. The authors report that the share of solutions passing the original tests stayed relatively stable as instructions were added, while the share passing the tests and satisfying every instruction dropped significantly. The tasks were single functions and the instructions were style rules, so the result shows how a requirement can vanish while tests stay green. It says nothing about how often that happens in Claude Code.

Sources: Findings of EMNLP 2025: When Instructions Multiply, arXiv 2509.21051 (StyleMBPP results)

Why a green suite and a clean diff both miss omissions

A diff shows what changed. An omission is a change that never happened, so there is nothing in it to review. A test suite has the same blind spot, because it checks the cases someone wrote down. Salesforce Engineering put it this way in a post about trusting AI-written code: "A green test suite could still leave a dangerous gap, confirming only that the written tests passed rather than that every requirement had become a test." Their answer was a matrix mapping each success criterion to supporting implementation and test evidence.

Sources: Salesforce Engineering: Why AI-Generated Code Is Easy but Engineering Trust Is Hard

Claude's own task list does not close the gap. The docs describe it as Claude's to-do checklist: items Claude created to plan multi-step work. An item that never made it onto that list is not marked as missing, so the requirements have to come from a file you wrote.

Sources: Claude Code docs: Interactive mode (task list)

Requirements that live only in the chat are at risk too. The memory docs say that if an instruction disappeared after compaction, it was given only in conversation, lives in a nested CLAUDE.md that has not reloaded yet, or is a path-scoped rule that has not matched a file since. They advise adding conversation-only instructions to CLAUDE.md. For a one-off task, a requirements file that the audit re-reads from disk keeps the list out of the chat.

Sources: Claude Code docs: How Claude remembers your project

A requirement ledger you can run yourself

The ledger is a table with one row per requirement, linking each one to the code that implements it and the test that proves it. It takes four steps, and none needs a plugin.

Step 1: write numbered requirements in a file

Write each requirement as one testable sentence with an ID, and split any sentence that joins two behaviors with "and" or "or" into two rows. Add an out-of-scope line so the audit can also spot extras. If you cannot write the list yet, the docs suggest letting Claude interview you ("Interview me until you could write the spec, then write it to SPEC.md") and building from a fresh session so the build starts with "a clean context and a written definition of done." For phrasing, see How to Write a Spec an AI Coding Agent Can Follow. A password-reset task might look like this.

Sources: Claude Code docs: Best practices (brainstorm first)

  • R1: A visitor can request a password reset from the login page.
  • R2: The reset link stops working 30 minutes after it is sent.
  • R3: An expired link shows the message "This link is no longer valid".
  • R4: A link that has already been used shows the same message.
  • R5: A reset request for an unknown email address shows the same confirmation as a known one.
  • Out of scope: changing the account email address.

Step 2: have Claude keep a ledger while it builds

Pro Tip

Paste this before the build: "Before writing code, copy every requirement from SPEC.md into LEDGER.md as a table with the columns id, requirement, status, where it lives (file and function), test that proves it, how it was checked. Keep each requirement word for word. Use only these statuses: done, partial, not done, deferred with a reason. Update the table as you finish each one. When you stop, print the whole table and list everything that is not done."

A missing row or an empty "where it lives" cell is an answer you can see, and the audit has something to compare. The ledger proves nothing yet, because it is Claude's own account. Weigh it the way you would weigh any "done" message.

Step 3: audit from a fresh context

Ask a reviewer that did not write the code to compare the work with SPEC.md alone, without reading the ledger. The docs say to write the review prompt yourself when you want a check against your plan instead of a general bug hunt, and give this one: "use a subagent to review the rate limiter diff against SPEC.md, check every requirement is implemented and the listed edge cases have tests, report gaps that affect correctness and skip style". Keep the "skip style" clause: the docs warn that "A reviewer that's asked to find problems will always find something."

Sources: Claude Code docs: Best practices (review against your spec)

Pro Tip

A longer version to adapt: "Review the work on this branch against SPEC.md. Do not read LEDGER.md. For each requirement ID, give a verdict (implemented, partial or missing), quote the file and line that implements it, and name the test that covers it. If you cannot find the code, write NOT FOUND. Then list every changed file that does not trace to a requirement. Report only gaps that affect correctness or the stated requirements." A subagent limited to the Read, Grep and Glob tools, as in the docs' code-improver example, has no Bash tool and cannot run git, so paste it the output of git diff --name-only main...HEAD (committed work) or git status --short (uncommitted work).

Be realistic about what this finds. By default a subagent is, in the docs' words, "the same model (or a smaller one) on the same task. You get parallelism or a fresh pair of eyes out of it, but not a smarter answer." The Montana self-critique results point the same way, since the same model missed the gaps that needed outside knowledge. Subagents also cost tokens, since each sends its own requests.

Sources: Claude Code docs: Best practices (subagents), arXiv 2602.02896

Step 4: check the wiring, then break something

For every row that names a new function or setting, search for callers. The command git grep --untracked -n -w validate_order -- . ':!tests' lists every mention of a name outside the tests directory (swap in your own test path), including in files not yet added to git. If the only hit is the definition, nothing calls it. We ran it with Git 2.54.0 in a throwaway repository containing a function that had a test but no caller, and it printed only the definition. A mention in a comment or string is not a call. Dynamic dispatch, decorators, registries and framework file conventions can hide a legitimate call from grep, so read each hit and treat a lone definition as a reason to look closer, not a verdict.

Then pick two or three rows and break them. Comment out the line the ledger points to and run only the test it names. If the test still passes, it does not prove the requirement. If it fails, read the failure: an import or syntax error from your edit proves nothing, while a failed assertion about the behavior the requirement describes does. Restore the file afterwards. This is a hand-run cousin of the technique in Mutation Testing for AI Code: Why Coverage Lies, applied to a few rows instead of the whole suite. Items that depend on outside systems, such as a message queue or a payment API, need a real run against the service.

Omission patterns and the check for each

Use this table to pick which check to spend time on. The patterns come from the reports, the Montana study and the docs above. The last two columns are our reasoning, not findings, and the order says nothing about frequency.

PatternWhat it looks likeWhy a green suite misses itCheck
Never startedNo code for one requirementNo test exists for it either, so nothing failsEmpty "where it lives" cell, or the reviewer writes NOT FOUND
PartialHappy path built; edge cases or one of several inputs missingTests were written for what was builtAsk for the edge cases per ID and confirm each has a named test
Defined, never usedA function or config key exists and nothing calls or reads itThe unit test calls the function directlyGrep for callers outside tests; change the setting and watch behavior
Dropped from a multi-part requestOne item is built and another is never mentionedNo test covers the dropped item, and the reply reads as completeNumber the parts and require a status for each
Outside the repoQueue, API or infrastructure setup is incompleteTests that mock the service pass, and the gap is not visible in application codeRun it against the real service and check by hand
Lost with the chatAn instruction given only in conversation can disappear after compactionNo file holds the requirementKeep requirements in a file and re-read it at audit time

Where the ledger stops working

  • It audits only what is written down. A requirement nobody stated leaves no row, so spend the time on the list first.
  • The ledger is a claim, and a same-model reviewer can share its blind spots. Sample-check a few rows yourself with the break test.
  • It costs tokens and attention. For a one-line fix, the diff is the requirement. Use the ledger when a task has several separable requirements.
  • Reviewers asked for gaps will report some, so ask only for gaps that affect the stated requirements.

Where TLM Forge fits

TLM Forge touches this problem at two points. For Medium and Deep tasks it shows you a design for your OK before building, an early moment to hold the design against your own requirement list. At the end, the phase-auditor, part of the final review wave, checks that what the spec promised was delivered and that out-of-scope items were respected. See Shows you the design and waits for your OK.

The limits are real. The phase-auditor is an LLM agent, so it can miss things. Light tasks get no review agents. Keep your own requirement list complete, because a requirement nobody wrote down gives any reviewer, human or LLM, nothing to check against. None of this replaces sampling a few ledger rows yourself.

Frequently asked questions

01Why does Claude Code skip requirements I gave it?

No single cause is established. User reports describe items dropped from multi-part messages, and the docs say instructions given only in conversation can disappear after compaction. Anthropic's docs also say Claude stops when the work looks done. Treat completeness as something to check against a numbered list.

02How do I check that Claude Code implemented every requirement?

Number the requirements in a file, have Claude keep a table of where each one lives and which test proves it, then have a fresh reviewer compare the code with the file alone. Finally, break two or three items by hand and confirm the named tests fail.

03Do passing tests mean the feature is complete?

No. Tests cover only the cases someone wrote. A requirement that never became a test, or code that was never built, produces no failure. In one benchmark of single-function tasks, test passing stayed relatively stable while the share that also met every added style instruction dropped significantly.

04Can Claude Code review its own work for missing requirements?

Partly. In one paper that used GPT-5, a same-model self-review fixed gaps that were visible in the code, such as a missing validation check, and made no progress on gaps needing outside knowledge, such as message queue configuration. Use a fresh context, and verify integration items by running them.

05Is a requirements ledger worth it for small changes?

Usually not. For a one-line fix the diff is the requirement. The ledger earns its cost when a task has several separable requirements, because a dropped item is easiest to miss there. Anthropic's docs also note that subagents cost tokens.

Ship AI-written code you can trust

TLM Forge is the missing process layer for Claude Code: a spec audit, independent multi-agent review, enforced TDD, and an adversarial red-team gate.

Get TLM Forge