How to Guardrail an AI Agent's CI/CD Pipeline

Where to place gates for an AI coding agent across pre-commit, PR, merge, and deploy, and why one LLM review comment is not a real gate.

An AI coding agent that can open a pull request or push a commit directly typically has reach into everything the pipeline can touch, including cloud credentials, git tokens, and signing keys. That agent needs more than a linter and a hopeful reviewer. It needs a pipeline with gates that a bad change physically cannot pass, placed at four points: before the commit leaves the agent, when the pull request opens, before the merge, and before the deploy. Each point stops a different failure, and skipping any of them leaves a hole an agent will eventually find.

Why "the AI reviewed it" is not a gate

A common pattern in agent-assisted pipelines is a step where a second model reads the diff and posts a comment saying it looks fine. That step is not a gate. It produces an opinion, not a pass or fail. Nothing stops the pipeline from proceeding regardless of what the comment says, nothing scores the finding against a threshold, and nothing prevents a rushed merge from happening anyway. A gate has to be mechanical: it either blocks the pipeline or it does not exist.

A single unenforced review step sits in front of a large amount of exposure. Guardrails for agent behavior are increasingly built as deterministic hooks that block an action outright, rather than as prompted instructions, because nothing guarantees a model will honor a written rule on a given turn the way a mechanical check does.

Sources: Agentic Coding Hooks: Deterministic AI Guardrails

• No enforcement: the pipeline continues whether the comment is positive or negative.
• No scoring: "looks good" and "has one nitpick" are treated identically.
• No accountability: nobody signed off, so nobody owns the decision.
• Easy to rubber-stamp: a model asked to review its own or a peer agent's work tends toward agreement, especially under a vague prompt.

Stage one: pre-commit, on the agent's machine

The cheapest place to catch a mistake is before it ever leaves the local environment. A pre-commit hook should run fast, deterministic checks that take seconds, not minutes, since anything slower gets skipped or bypassed under pressure.

• Secret scanning with a lightweight, offline scanner. Gitleaks matches against a ruleset of over a hundred known secret formats, including AWS keys, GitHub tokens, and private keys, and is fast enough to run on every commit.
• Linting and formatting so style noise never reaches a human or a later gate.
• A fast subset of unit tests scoped to the changed files, not the full suite.

Gitleaks and TruffleHog are the two most widely used open-source secret scanners; Gitleaks is built for speed at the pre-commit stage, while TruffleHog goes further by actually calling the credential's origin service to verify whether a detected key is live, which is better suited to a slower CI stage than a commit hook.

Sources: Gitleaks, TruffleHog

None of this stage is trustworthy on its own for an agent that can push directly. A local hook is a file on disk, and an agent that can run a shell command can also run one with the hook flag disabled, or push through an API path that never triggers the hook at all. Treat pre-commit as a courtesy that saves round trips, not as a control. Every check that matters for safety has to be repeated somewhere the agent does not control the execution path, which is exactly what the next three stages are for.

Stage two: when the pull request opens

The moment an agent opens a PR is the first point where the full diff exists against the target branch, and where automated checks can run against everything: not just the changed lines, but the changed dependency graph and the changed blast radius.

• Full secret scan with verification. Run TruffleHog or GitHub secret scanning across the complete diff, not a sample, and turn on push protection so a detected secret blocks the push before it ever lands in history rather than only alerting after the fact.
• Dependency-diff review. Every added or updated package is untrusted input from outside the repository. GitHub's dependency review flags known vulnerabilities in new or updated dependencies, including transitive ones pulled in through a lockfile, and can be set to fail the check on discovery.
• Static analysis tuned to how agents actually fail. Generic scanners like Semgrep and CodeQL catch classic bug classes, but an agent repeats its own failure modes: swallowed exceptions, disabled certificate checks, hardcoded fallback credentials, string-built SQL. Custom rule packs targeting those specific patterns catch what a generic ruleset misses.
• Blast-radius labeling. Classify the changed files by what they touch: authentication, billing, infrastructure-as-code, database migrations, or low-risk paths like docs and tests. This label decides how strict the next stage needs to be.

GitHub's dependency review can also surface OpenSSF Scorecard data for changed dependencies directly in the pull request, giving a maintenance and security-practice signal for a package before it is merged in, not after an incident traces back to it.

Sources: OpenSSF Scorecard info in the Dependency Review Action

Security teams that write rules for agent-generated code specifically have found that generic guidance underperforms: narrowly scoped rules with concrete code examples and explicit decision logic catch more than a broad "write secure code" instruction, because the failure is contextual, not conceptual.

Sources: How to Write Skills That Make Your AI-Generated Code More Secure, Semgrep

Stage three: before the merge

This is the stage that has to be unconditionally mechanical, because it is the last checkpoint before the change becomes part of the shared history. Two GitHub primitives do most of the enforcement work here: required status checks and CODEOWNERS-based required review.

• Test coverage threshold. Fail the merge if coverage on changed lines drops below a set floor, not just if any test fails. An agent can pass a thin, tautological test it wrote itself; a coverage floor on the diff forces breadth.
• Full test suite green, not a cached or partial result.
• Required human review scaled to blast radius. Route high-risk paths, auth, payments, infrastructure-as-code, migrations, through CODEOWNERS so those changes automatically require a specific reviewer, while a docs-only change does not need the same friction.
• Branch protection that the agent's own identity cannot bypass: no self-merge, no force-push to the protected branch, no admin override left enabled by default.

CODEOWNERS lets a repository encode review tiers by blast radius so the platform enforces them instead of a policy document: a rule can require that changes under an auth or billing path always get a review from a specific team before merge, combined with branch protection that makes that review mandatory rather than advisory.

Sources: About code owners, GitHub Docs

The coverage floor should apply to the lines the agent actually touched in this change, not the repository average. A codebase can sit at a healthy overall coverage number while a new file the agent added has none, because the average absorbs it. Scoping the threshold to the diff closes that gap, and it is the same reason required status checks exist as a distinct branch protection setting: a check that exists but is not marked required can fail silently and the merge button stays green anyway.

Stage four: before the deploy

A change can pass every prior gate and still be dangerous to ship immediately, particularly if it touches infrastructure or a migration. The pre-deploy gate is about provenance and blast radius at the moment of release, not about re-litigating the code review.

• Build provenance: know, with a signed attestation, which commit and which build process actually produced the artifact being deployed.
• A deploy-time blast-radius check: does this change touch a migration, a secret rotation, or an IaC file that affects more than the service being deployed? If so, route to a slower, more supervised deploy path.
• A tested rollback path for anything above the low-risk tier. If the agent cannot articulate how the change is reverted, that alone is a reason to hold it.

Require at least a signed attestation from dedicated build infrastructure before deploy, which is SLSA Level 2. SLSA's build track defines provenance in progressive levels: simply having provenance documentation at Level 1, that signed attestation at Level 2, and isolation strong enough that one build cannot tamper with another at Level 3. Each level closes off a more sophisticated way of substituting a tampered artifact for the one that was actually reviewed.

Sources: SLSA Build Track Levels

Where this fits with TLM Forge

The four stages above are pipeline-level plumbing: secret scanners, dependency-diff tools, branch protection, and provenance checks that apply to any team shipping code, agent-written or not. TLM Forge operates one layer up, at the point where the agent's actual work gets judged. It runs a scored merge gate that blocks until every unresolved critical finding clears, backed by reviewers launched in fresh context specifically to attack the change, a threat-modeler at design time and a red-team pass on the diff, rather than a single cooperative pass that assumes the work is basically right. It also enforces test-driven development with the full suite run captured as evidence, so "tests pass" is a reproducible artifact and not a claim to take on faith. None of that replaces a secret scanner or a dependency-diff check; it is the part of the pipeline concerned with whether the change itself was built and reviewed correctly, sitting on top of the mechanical checks described above.