Why Does Version Pinning Matter for Python Package Security? The LiteLLM Lesson
I got a panicked Slack message at 2 AM: “Our OpenAI API keys were compromised. We found them on a paste site.”
The incident response team traced it back to a single line in our requirements.txt:
litellm>=1.0.0That >= cost us $12,000 in unauthorized API usage and two weeks of security remediation.
The Problem: A Single Character Changed Everything
I’d been using lower-bound version constraints for years. It seemed like best practice:
# My usual approachrequests>=2.28.0flask>=2.0.0litellm>=1.0.0“Always get the latest compatible version,” I thought. “Security patches, bug fixes, improvements—why wouldn’t I want those?”
Then LiteLLM version 1.82.0 shipped with malicious code.
What Happened with LiteLLM
LiteLLM is a popular Python package that provides a unified interface to multiple LLM APIs (OpenAI, Anthropic, AWS Bedrock, etc.). It’s used by thousands of projects, including enterprise applications handling sensitive API credentials.
In December 2024, the package was compromised:
Day 1 → Attacker gains access to LiteLLM PyPI accountDay 2 → Malicious versions 1.82.0-1.82.3 publishedDay 2 → Code exfiltrates API keys to attacker serverDay 3 → Community discovers and reports compromiseDay 3 → Malicious versions removed from PyPI
Impact: 47,000+ users potentially affected API keys for OpenAI, Anthropic, AWS exposedThe malicious code was subtle—it only activated in production environments, avoiding detection in development and testing. It captured every API key passed to LiteLLM and sent them to an attacker-controlled server.
The 59% Vulnerability Statistic
Here’s what shook me: security researchers analyzed 2,337 packages depending on LiteLLM. The breakdown:
┌─────────────────────────────────────────┬────────┬─────────────┐│ Version Constraint Type │ % │ Status │├─────────────────────────────────────────┼────────┼─────────────┤│ >=1.0.0 (lower-bound only) │ 59% │ COMPROMISED ││ No constraints at all │ 12% │ COMPROMISED ││ >=1.0.0,<2.0.0 (broad upper bound) │ 16% │ COMPROMISED ││ ==1.81.0 (exact version) │ 12% │ SAFE ││ Other pinned versions │ 1% │ SAFE │└─────────────────────────────────────────┴────────┴─────────────┘
88% of packages were vulnerableOnly 13% were protected59% of projects—over 1,300 packages—used the exact pattern I had: litellm>=1.0.0. Every one of them automatically pulled the compromised version on their next pip install or CI build.
Why My “Best Practice” Was Wrong
I had three misconceptions about version constraints:
Misconception 1: “Newer is always better”
New version → Bug fixes → Security patches → Must be good!Reality: New versions can introduce:
- Malicious code (supply chain attacks)
- Breaking changes you didn’t test
- New dependencies with their own risks
Misconception 2: “Upper bounds protect me”
litellm>=1.0.0,<2.0.0 # I thought this was safeThe malicious 1.82.0 fell perfectly within this range. Upper bounds protect against major version changes, not minor version compromises.
Misconception 3: “I can review before upgrading”
Developer runs: pip install --upgradePackages updateDeveloper reviews changesDeveloper commits new requirementsCI pipeline runs: pip install -r requirements.txtFresh environment, no cachepip resolves >=1.0.0 to latest (1.82.0)Build succeeds, malicious code deployedNo human review involvedThe Fix: Exact Version Pinning
I changed every security-sensitive dependency to exact versions:
# Loose constraintslitellm>=1.0.0openai>=1.0.0boto3>=1.26.0stripe>=5.0.0# Exact pins for security-sensitive packageslitellm==1.81.0openai==1.12.0boto3==1.34.0stripe==5.5.0But this wasn’t enough. I realized my transitive dependencies weren’t pinned.
Deeper Problem: Transitive Dependencies
LiteLLM depends on other packages:
litellm==1.81.0├── openai>=1.0.0 # Not pinned!├── aiohttp>=3.8.0 # Not pinned!├── requests>=2.28.0 # Not pinned!└── ... (dozens more)Even with litellm==1.81.0, the transitive dependencies could still pull in compromised versions if their maintainers specified loose constraints.
Solution: Lock Everything
I switched to pip-tools for complete dependency locking:
# Direct dependencies onlylitellm==1.81.0openai==1.12.0flask==3.0.0stripe==5.5.0# Install pip-toolspip install pip-tools
# Generate fully resolved requirements.txtpip-compile requirements.in
# This pins ALL transitive dependencies# This file is autogenerated by pip-compilelitellm==1.81.0 # via -r requirements.inopenai==1.12.0 # via -r requirements.inaiohttp==3.9.1 # via litellmaiosignal==1.3.1 # via aiohttpasync-timeout==4.0.3 # via aiohttp# ... every single dependency pinnedNow my builds are reproducible. Same versions, every time, across all environments.
For Production: Private PyPI Mirror
For production systems, I implemented a private mirror:
[global]index-url = https://pypi.yourcompany.com/simple/timeout = 30┌─────────────────────────────────────────────────────────────┐│ Private PyPI Mirror │├─────────────────────────────────────────────────────────────┤│ ││ Upstream PyPI ──► Mirror ──► Your Production ││ │ ││ ├─ Lead time: 7 days ││ ├─ CVE scanning ││ ├─ Manual approval required ││ └─ Only approved versions cached ││ ││ Benefits: ││ • Malicious versions blocked before reaching you ││ • Time to respond to security incidents ││ • Audit trail of all dependencies ││ • Faster builds (cached packages) │└─────────────────────────────────────────────────────────────┘This creates a buffer between new package releases and production deployment.
When to Pin vs. When to Allow Updates
I developed a decision matrix:
┌─────────────────────────────────┬────────────────┬─────────────────┐│ Package Type │ Pin Strategy │ Why │├─────────────────────────────────┼────────────────┼─────────────────┤│ Handles credentials/API keys │ EXACT VERSION │ Supply chain ││ (litellm, boto3, stripe) │ ==1.2.3 │ attack risk │├─────────────────────────────────┼────────────────┼─────────────────┤│ Frameworks with breaking changes│ EXACT VERSION │ Stability ││ (flask, django, fastapi) │ ==1.2.3 │ │├─────────────────────────────────┼────────────────┼─────────────────┤│ Pure utilities, no network │ RANGE OK │ Low risk ││ (python-dateutil, pydantic) │ >=1.2.0,<2.0.0 │ │├─────────────────────────────────┼────────────────┼─────────────────┤│ Internal packages │ EXACT VERSION │ Control ││ │ ==1.2.3 │ │└─────────────────────────────────┴────────────────┴─────────────────┘
Rule of thumb: "Anything that touches API credentials deserves a pinned exact version"CI/CD Pipeline Changes
I also updated our CI pipeline:
- name: Install dependencies run: pip install -r requirements.txt # Fresh install every time, no caching- name: Set up Python uses: actions/setup-python@v5 with: python-version: '3.11' cache: 'pip'
- name: Install dependencies run: | pip install pip-tools pip-sync requirements.txt # Uses cache, fails if lockfile doesn't match
- name: Verify lockfile is current run: | pip-compile requirements.in --output-file requirements.txt --check # Ensures requirements.in and requirements.txt are in syncThe Reddit Insight
A comment on the Reddit thread about LiteLLM crystallized the issue:
“Anything that touches API credentials deserves a pinned exact version.”
This seems obvious in retrospect. We carefully protect our credentials with environment variables, secret managers, encrypted vaults… then pass them to a package that auto-updates to the latest version from the internet every build.
Key Takeaways
- 59% of LiteLLM dependents were compromised because of
>=version constraints - Only 13% were protected—those using exact version pinning
- Upper bounds don’t protect against minor version compromises
- Transitive dependencies matter—pin them too, or use a lockfile
- Private mirrors add a safety buffer for production systems
What I Do Now
1. Pin exact versions for security-sensitive packages2. Use pip-tools for transitive dependency locking3. Commit requirements.txt (lockfile) to version control4. Review all dependency updates before merging5. Implement private PyPI mirror for production6. Add lead time (7+ days) between release and deploymentThe $12,000 lesson: version constraints aren’t just about compatibility—they’re a security boundary. That >= symbol handed our API keys to an attacker.
Further Reading
For a deeper dive into Python dependency security:
- The pip-tools documentation has excellent examples of reproducible builds
- PyPI’s security best practices guide covers supply chain protection
- The Python Packaging User Guide explains dependency resolution
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
- 👨💻 LiteLLM Security Incident Analysis
- 👨💻 pip-tools Documentation
- 👨💻 PyPI Security Best Practices
- 👨💻 Python Dependency Management Guide
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments