Skip to content

supply-chain

1 post with the tag “supply-chain”

The 40-Minute Supply Chain Attack That Exposed 434,000 CI/CD Pipelines

The malicious packages behind the largest AI supply chain breach of 2026 survived on PyPI for only 40 minutes (TechJuice). The fallout is still being counted. On August 11, threat intelligence firm CloudSEK published a report linking more than 2,500 organizations and roughly 434,000 software pipelines to the compromise of LiteLLM (CloudSEK via PR Newswire). Independent analysis from Hudson Rock confirmed the scale the next day (Hudson Rock).

LiteLLM is an open-source proxy that gives applications a single API for many large language model providers (CyberInsider). Teams run it as the gateway between their code and models from OpenAI, Anthropic, and others. The library is downloaded more than 95 million times per month (CyberInsider). That reach is why it became a target. An environment running LiteLLM holds API keys, cloud credentials, and configuration files by design.

The attack did not start with LiteLLM. It started with Trivy, the open-source vulnerability scanner (Hudson Rock). TeamPCP, the group behind the campaign, first compromised Trivy’s GitHub Actions pipeline (CyberInsider). The group used an automation token that was rotated but never fully revoked. That gap gave them a 20-day window to force-push malicious code over Trivy’s version tags (TechJuice).

LiteLLM’s own CI pipeline used Trivy to scan its builds. The poisoned scanner had legitimate read access to the build runner. The attackers used that access to exfiltrate LiteLLM’s PyPI publishing tokens (Hudson Rock). With those tokens they published two poisoned releases, versions 1.82.7 and 1.82.8, to PyPI (CyberInsider). The malicious packages were pulled after about 40 minutes (TechJuice). Version 1.82.6 was the last clean release (Endor Labs).

The injection was small and surgical. Twelve lines of obfuscated code were added to a single file, litellm/proxy/proxy_server.py, during the wheel build (CyberInsider). The code decoded a base64 payload and launched it through a Python subprocess when the module was imported. Version 1.82.8 escalated the attack. It added a .pth startup file that runs the payload every time Python starts, even when LiteLLM is never imported (CyberInsider).

The payload harvests a wide credential set. It grabs SSH keys, AWS, GCP and Azure credentials, Kubernetes secrets, environment files, database configurations, and cryptocurrency wallets (CyberInsider). Stolen data is encrypted, packed into a file named tpcp.tar.gz, and exfiltrated to an attacker-controlled domain (CyberInsider). When that path fails, the malware creates a public repository in the victim’s own GitHub account. It uploads the stolen data as a release asset (TechJuice). The payload also moves laterally in Kubernetes. It deploys privileged pods that mount the host filesystem and install a persistent backdoor registered as a systemd service named “System Telemetry Service” (CyberInsider).

CloudSEK identified more than 2,500 organizations potentially impacted. The list spans technology, finance, telecom, cybersecurity, manufacturing, and logistics (CloudSEK via PR Newswire). Hudson Rock obtained a 153GB archive of the stolen data containing 433,909 files. It attributed 118,829 CI runner dumps to 2,488 corporate domains (Hudson Rock). Named victims include NVIDIA, Samsung Electronics, Cisco Systems, Siemens, S&P Global, ServiceNow, and Deloitte (Unite.AI). The trace also surfaced Boeing, Orange, and Roku (TechJuice). The exposed material covers AWS secrets, GitLab identities, Salesforce credentials, Slack tokens, Azure secrets, SSH keys, and AI provider API keys (TechJuice).

An AI gateway is the richest credential store in a modern stack. Every LLM provider key, cloud secret, and pipeline token flows through it. A single poisoned release in that position turns months of build history into an attacker’s keychain. The 40-minute window on PyPI is the core lesson: exposure time no longer measures damage. The packages were published in March 2026, yet organizations are only learning of their exposure in August (Unite.AI).

  1. Revoke, do not just rotate. The entry token was rotated but never revoked (TechJuice). Rotation leaves the old credential alive. Revocation kills it.
  2. Pin with hashes. A lockfile with integrity hashes blocks a malicious release from installing, even when it reaches the index. This is the single cheapest control in the chain.
  3. Separate publish access from build access. The scanner that reads your repo should not also hold your package-publishing tokens (Hudson Rock).
  4. Audit secrets continuously. Environment variables leak into runner dumps and public repos (TechJuice). Scan for them on every run, not once a quarter.
  5. If you ran LiteLLM 1.82.7 or 1.82.8, act now. Treat every credential in that environment as compromised and rotate them. The malware targeted .aws/credentials and .kube/config specifically (TechJuice).

The pattern is familiar to anyone who read our breakdown of credential theft through AI developer tools. The tool that has access becomes the target. The LiteLLM breach just proved it at the scale of the entire AI build ecosystem.