Untrusted issues, docs, and tool output can steer an AI coding assistant off course, and its generated code can repeat the same flaw. How to guard both.
An AI coding assistant reads more than the prompt you type. It pulls in file contents, terminal output, fetched web pages, and text from issues, pull requests, and linked documentation, then folds all of it into the same reasoning process. Any one of those sources can contain text that looks like an instruction rather than reference material. That is the basic shape of prompt injection against a coding assistant: content the assistant is meant to treat as data actually contains a command, and if nothing separates the two, the assistant may act on it.
This produces two separate risks worth treating as distinct problems. The first is the assistant itself getting steered by something it read, which can push it toward writing code with a weakened check, a hidden backdoor, or a leaked secret. The second is that the code the assistant produces can carry the identical flaw forward: a function that takes user input and passes it, unexamined, into a shell command or into another LLM prompt. Neither problem needs anything exotic to defend against. Both come down to one habit: never let untrusted text control what a system does next.
Two Attack Surfaces, One Root Cause
It helps to separate where the risk sits before deciding how to close it. The first surface is upstream, content flowing into the assistant while it works. The second is downstream, content flowing into code the assistant has already written and shipped. Both trace back to the same root cause: a failure to keep instructions and data in separate channels. That means the same mitigation pattern applies to each, just applied at a different point in the pipeline.
When the Assistant Reads Untrusted Content
Coding assistants routinely pull in content nobody on your team wrote. An issue can contain multi-paragraph text formatted exactly like a task description, because that is what it is meant to look like to anyone, or anything, reading it. A fetched documentation page can include text placed there specifically for a browsing tool to pick up. Output from a shell command, a test run, or an API response is still just a string, and a string does not stop reading like an instruction just because it arrived through a trusted-looking pipe.
- Issue trackers and pull request descriptions: anyone with write or comment access can shape what the assistant reads as "the task"
- Fetched web pages and third-party docs: written by someone outside your organization and never reviewed by you before the assistant sees them
- Tool and command output: build logs, test failures, and API responses get treated as ground truth, but they are still just text
- Sample code copied from forums, READMEs, or package descriptions: the code and the prose around it enter the same context together
Treat every piece of content the assistant did not receive directly from you, in your own prompt, as data to evaluate, never as an instruction to execute. If your process cannot tell the two apart, assume the more cautious reading.
When Generated Code Repeats the Same Mistake
The second angle is easy to miss because it looks like an ordinary bug rather than a security concept. An assistant asked to add a feature that summarizes user-submitted text with an LLM, or that runs a user-supplied command, will often produce a working first draft with no separation between instruction and data at all. User input gets formatted directly into the prompt sent to the model, or concatenated directly into a shell command string. It passes every functional test, because functional tests do not submit adversarial input, so the gap stays invisible until someone does.
- A prompt assembled with an f-string that drops raw user text straight into the system instructions
- A shell command built by string concatenation instead of an argument array or a subprocess API that never touches a shell interpreter
- A database query built with string formatting instead of bound parameters
- A "safe mode" flag that an attacker-controlled string can simply include as part of its own payload
Ask the assistant directly: "Show me every place user input reaches an LLM prompt or a shell command in this diff, and how each one is isolated from instructions." Naming the check produces a materially different answer than a generic "review this for security issues."
Closing Both Gaps
The fix on both sides is structural, not just diligence. Wherever untrusted text enters a prompt, wrap it in a clearly labeled field and instruct the model, in the trusted system portion of the prompt, to treat that field as content to summarize or describe, never as a command to follow. Wherever untrusted text reaches a shell, use an argument list or a language binding that never passes through a shell interpreter, instead of assembling a command string by hand. The same discipline applies to the assistant's own tool access: give it only the file system, network, and execution rights the task in front of it actually needs, and it has little to act on even if an injection attempt gets through.
Structural isolation reduces the odds of a mistake, but it does not catch every instance across a large diff or a fast-moving feature branch. That is what an adversarial review step is for: a pass that reads the code from the position of someone trying to break it, specifically hunting for places where untrusted input reaches a prompt, a shell, or a query without a boundary in between. A red-team style review run after correctness has already been checked catches a class of bug that a correctness-focused reviewer has no reason to look for, and it is the same discipline behind adding explicit guardrails around any code path that touches untrusted input by design.
Unguarded vs. Guarded Handling of Untrusted Input
| Untrusted input source | Unguarded handling | Guarded handling |
|---|---|---|
| Issue or PR text | Assistant reads embedded commands in the body as instructions to follow | Body is read as reference content only; the user's own prompt is the sole instruction channel |
| Fetched web page or doc | Page content is pasted into the prompt with no boundary marker | Content is wrapped in explicit delimiters and labeled as untrusted reference material |
| Tool or command output | Output feeds straight back into the next reasoning step unfiltered | Output is treated as data, reviewed or summarized before reuse, never as new instructions |
| User input into generated shell code | String-concatenated directly into a shell command | Passed as an argument list or through an API that bypasses the shell entirely |
| User input into a generated LLM prompt | Interpolated directly into the system or instruction portion of the prompt | Isolated in a clearly delimited data field, with the model told never to treat it as commands |
Keep the Standard From Resetting Every Session
A single review catches what is in front of it that day. The harder problem is applying the same standard every time a new feature touches user input, especially across sessions where a model starts with no memory of what it was told to watch for last time. Which fields are always parameterized, which tools never touch a shell, which content types are always treated as untrusted: conventions like these need to persist somewhere the next session, and the next reviewer, can actually reference, instead of being re-explained from scratch. A private, portable memory layer like MemX is one way to carry that kind of project-specific context forward across sessions, so the rule does not reset every time a new conversation starts.
Frequently asked questions
01Is prompt injection the same thing as a jailbreak?
No. A jailbreak targets the model's own safety behavior, trying to get it to produce content it would normally refuse. Prompt injection targets an application built around the model, hiding instructions inside content the application feeds to the model so it acts on those instructions instead of, or in addition to, the user's actual request.
02Can keyword filtering alone stop prompt injection?
Not reliably. Filtering catches obvious phrasing and misses anything worded differently, so treat it as one weak layer, not the defense. The reliable fix is structural: keep untrusted content in clearly labeled data fields, keep instructions in a channel only you control, and give tools the minimum access they need.
03Does this only matter for public repositories?
No. Private repositories still pull in third-party dependencies, documentation, and issue text from anyone with access, and internal tools often fetch external content on the assistant's behalf. The trust boundary is about where content originated, not whether the repository is public.