When AI-Written Code Breaks in Production

AI code is slow to debug in production not because it fails more, but because nobody kept the intent. Fix it with provenance at authoring time.

AI-written code is not slower to fix in production because it fails more often. It is slower to fix because no one on the team holds the intent behind it. When a service pages at 3am and the offending change came from a coding agent that ran three weeks ago, the responder is not reading a bug, they are reverse-engineering a decision nobody remembers making. Root-cause analysis stalls on the why, not the what, and the why left with the session that produced the code.

The fix belongs at authoring time, not in the incident channel. Tag every AI-authored change with machine-readable provenance, keep the spec and the agent plan linked to the diff so a responder reads intent instead of guessing it, and instrument the exact seams where generated code is weakest: concurrency, cache expiry, edge cases, and error handling. None of that is heroics at 3am. It is a discipline you pay for once, when the code is written, so that the person paged later inherits signal instead of a mystery.

The pressure behind this is measured. The 2024 DORA report estimated that for every 25 percent increase in AI adoption, software delivery stability falls by roughly 7.2 percent, even as individual output rises. GitClear's analysis of 211 million changed lines found that 2024 was the first year copy-pasted code exceeded refactored code, and that code churn, the share of lines rewritten within two weeks, roughly doubled from its pre-AI baseline. More code, shipped faster, revised more, and increasingly written by an author who is not a person. The odds that the change paging you tonight is AI-authored, and that its author is unavailable, keep climbing.

Sources: DORA | Accelerate State of DevOps Report 2024, GitClear: AI Copilot Code Quality, 2025 research (devclass summary), GitClear: AI Copilot Code Quality 2025 report (churn data)

This post argues that the cure is a discipline applied when code is authored, and that thesis points somewhere commercial: we build TLM Forge, a process layer that gates AI-written changes before they ship. So read the vendor-neutral parts as the load-bearing claims, and read the one product section knowing where it comes from.

Why AI code is slower to diagnose, not more likely to break

The expensive part of an incident is comprehension, not the keystroke that fixes it. DORA renamed its recovery metric from mean time to restore to failed deployment recovery time, but the clock still runs mostly on understanding: what changed, what it was supposed to do, and why it does something else under load. For code a human wrote, that understanding has a trail. Git blame points at a name, the name points at a pull request, the pull request has a description and a review thread, and often the author is one Slack message away. The intent is recoverable because it was distributed across people and artifacts.

Sources: DORA: the software delivery performance metrics (four keys)

AI-authored code breaks that chain in a specific place. The diff is legible, the tests may even be green, but the reasoning that produced it lived in a prompt, a plan, and a model context that no one saved. Git blame resolves to whoever clicked merge, not to whoever decided the cache should expire after sixty seconds or that this error should be swallowed rather than raised. The responder can read every line and still not know which lines are load-bearing and which are the model padding a function it did not fully reason about. That gap between reading code and understanding intent is where the extra minutes go.

This is not an argument that AI code is worse. Plenty of it is fine. The argument is narrower: identical code is more expensive to operate when its provenance is missing, because incident response is an act of reconstruction and generated code arrives with its reconstruction data already deleted. Restore the data at the source and the code stops being uniquely hard to diagnose.

Provenance: tag every AI-authored change at the source

Make AI authorship a machine-readable fact on every commit, not a thing people try to remember. Git already supports this through trailers, the RFC-822-style key-value lines at the end of a commit message that git interpret-trailers can parse out programmatically. A Co-authored-by trailer naming the tool and model, or a custom Generated-by trailer, turns "was this AI-written?" from tribal knowledge into a query. Some coding agents already emit an authorship trailer by default, and the point is to keep it and standardize it rather than strip it on squash.

Sources: git-interpret-trailers documentation

Provenance pays off precisely when an incident is in flight. A responder who can filter the last deploy for AI-authored commits knows immediately which changes carry a thinner human review trail and deserve first suspicion. A retro that can query provenance across a quarter can tell whether a class of incidents clusters around generated code in a particular subsystem. Neither is possible if authorship is a vague memory. Capture it once, structurally, and it stays queryable.

• Commit trailer: a Co-authored-by or Generated-by line naming the tool and model, preserved through squash and rebase so git interpret-trailers --parse can extract it later.
• Pull request label: an ai-authored or ai-assisted label so the change is filterable in the review queue and in incident tooling that reads PR metadata.
• Scope note: which parts of the diff were generated versus hand-edited, because a half-AI change hides its risk if the whole PR is tagged with one flag.
• Model and prompt reference: a stable link or ID pointing at the session that produced the change, so intent is one click away rather than lost.

Keep the spec and the plan linked to the diff

A tag tells a responder that code is AI-authored. It does not tell them what the code was meant to do. Close that gap by linking the spec the agent worked from and the plan it produced directly to the diff, so the pull request carries intent alongside implementation. When the incident is "the rate limiter is rejecting valid traffic," the responder wants to read the sentence that said "reject at 100 requests per minute per token, fail open on Redis timeout" far more than they want to infer that rule from the code. Intent stated up front is worth more at 3am than intent excavated line by line.

The hard part is that agent context is ephemeral. The spec, the interface it agreed to, and the reason a value was chosen live in a session that closes and takes the reasoning with it. A private, persistent memory layer such as MemX keeps those decisions durable across sessions, so the intent behind a change survives the agent that made it and is still readable by the human who has to defend it in production months later. Provenance without retained intent is a name tag with no story attached.

Instrument the seams where AI code is weakest

Spend your observability budget where generated code fails most, not uniformly across the codebase. Language models are strongest at the happy path and measurably weaker at the parts a request touches only under stress: concurrent access, cache expiry and eviction, boundary conditions, and error handling that has to do the right thing when a dependency is down. These are the seams that pass a code review because they read plausibly and pass a test suite because the suite exercises the common case. They surface only in production, under the exact conditions no one instrumented.

Put telemetry on those seams deliberately. OpenTelemetry defines a span as a single unit of work with timing, context, and outcome, and a trace as the tree of spans that reconstructs a request's full path. Wrapping a cache lookup, a lock acquisition, or a fallback branch in its own span means the seam reports its own behavior instead of hiding inside an aggregate latency number. When the incident lands, the responder sees which branch executed and how long it took, rather than inferring it from a diff whose author is gone.

Sources: OpenTelemetry: observability primer (traces and spans)

• Concurrency: span the lock acquisition and critical section, and count contention, so a deadlock or a lost update shows up as a signal instead of a stall.
• Cache TTL and eviction: record hit rate, evictions, and the age of served entries, because a generated default TTL is the classic silent-until-3am bug.
• Edge cases: emit a metric on the boundary branches (empty input, max size, retry exhaustion) that reviews wave through and load finds.
• Error handling: log and span the catch blocks, especially any that swallow an error, so a failure that was quietly absorbed still leaves a trace.

The discipline belongs before merge, not at 3am

Every fix above is cheap when applied at authoring time and nearly impossible to apply retroactively. You cannot recover a prompt you did not save or a span you did not emit once the incident is live. So the practical move is to make provenance, linked intent, and seam instrumentation part of the gate a change passes before it merges, rather than a cleanup task that never gets prioritized. That is the gap TLM Forge is built for: a spec audit forces the intent to exist and stay attached before any code is written, independent review agents examine the seams where generated code is weakest, and an adversarial convergence gate blocks promotion until critical issues reach zero instead of until CI turns green. It does not stop AI code from ever breaking. It makes the code that breaks legible to the person who has to fix it.

This is a different job from using AI to help you debug. Debugging with AI points a model at a failure you are already staring at. This post is the opposite discipline: preparing AI-authored code, before it ships, so that any responder can diagnose it fast. The two pair well, and both depend on the same foundation, tests that actually exercise the seams. Testing AI-generated code covers why a green suite on generated code is a weaker signal than it looks, which is exactly why the concurrency and error paths need instrumentation rather than trust.

AI coding tools are a real speed gain, most generated code is serviceable, and none of this is a reason to write less of it. The claim is narrower: generated code costs more to operate when it ships without provenance and without signal on its weak seams, and that cost lands on whoever is on call. Teams that tag authorship, keep intent attached, and instrument the concurrency, cache, edge-case, and error paths turn a 3am reconstruction project back into an ordinary bug.