AI in CI/CD pipelines can be tricked into behaving badly

AI agents embedded in CI/CD pipelines can be tricked into executing high-privilege commands hidden in crafted GitHub issues or pull request texts.

Researchers at Aikido Security have traced the problem back to workflows that pair GitHub Actions or GitLab CI/CD with AI tools such as Gemini CLI, Claude Code Actions, OpenAI Codex Actions or GitHub AI Inference. They found that unsupervised user-supplied strings such as issue bodies, pull request descriptions, or commit messages, could be fed straight into prompts for AI agents in an attack they are calling PromptPwnd.

Depending on what the workflow lets the AI do, this can lead to unintended edits to repository content, disclosure of secrets, or other high-impact actions.

“AI agents connected to GitHub Actions/GitLAb CI/CD are processing untrusted user input, and executing shell commands with access to high-privilege tokens,” the researchers wrote in a blog post about PromptPwnd. They said they reproduced the problem in a test environment, and notified the affected vendors.

The researchers recommended running a set of open-source detection rules on suspected GitHub Action .yml files, or using their free code scanner on GitHub and GitLab repos.

Aikido Security said that Google had patched the issue in Gemini CLI upon being informed; Google did not immediately respond to a request for information about this.

Why PromptPwnd works

PromptPwnd exploits become possible when two flawed pipeline configurations occur together: when AI agents operating inside CI/CD workflows have access to powerful tokens (like GITHUB_TOKEN, cloud-access keys), and their prompts embed user-controlled fields.

Prompt injection becomes easier with such a setup, the researchers explained. An attacker can simply open an issue on a public repository and insert hidden instructions or seemingly innocent comments that double as commands for the model to pick. “Imagine you are sending a prompt to an LLM, and within that prompt, you are including the commit message,” the researchers said. “If that commit message is a malicious prompt, then you may be able to get the model to send back altered data.” The model’s response, if used directly inside commands to tools within CI/CD pipelines, can manipulate those tools to retrieve sensitive information.

Aikido Security demonstrated this in a controlled environment (without real tokens) to show that Gemini CLI could be manipulated into executing attacker-supplied commands and exposing sensitive credentials through a crafted GitHub issue. “Gemini CLI is not an isolated case. The same architecture pattern appears across many AI-powered GitHub Actions,” the researchers said, adding that the list included Claude Code, OpenAI Codex, and GitHub AI Inference.

All of these tools can be tricked (via issue, pull-request description, or other user-controlled text) into producing instructions that the workflow then executes with its privileged GitHub Actions token.

Mitigation plan

Aikido has open-sourced detection rules via their “Opengrep” tool that allows developers and security teams to scan their YAML workflows automatically, revealing whether they feed untrusted inputs into AI prompts.

The researchers said that only a subset of workflows have confirmed exploit paths so far, and that it is working with several other companies to address the underlying vulnerabilities. Some workflows can only be abused with collaborator-level access, while others can be triggered by anyone who files an issue or pull request.

Developer teams are advised to restrict what AI agents can do, avoid piping untrusted user content into prompts, treat AI output as untrusted code, and contain damage from compromised GitHub tokens.

Aikido Security said its code scanner can help flag these vulnerabilities by detecting unsafe GitHub Actions configurations (including risky AI prompt flows), identifying over-privileged tokens, and surfacing insecure CI/CD patterns via infrastructure-as-code scanning.

There are other best practices for securing CI/CD pipelines that enterprises can adopt, too.

This article first appeared on Infoworld.