How to Migrate from LiteLLM to Bifrost: Step-by-Step Guide

Mar 26, 2026

Problem

The LiteLLM supply chain attack in March 2026 exposed critical vulnerabilities in Python-based AI gateways. I needed to migrate my LLM infrastructure to a more secure alternative without rewriting all my application code.

After evaluating several options, I chose Bifrost. The main reasons:

Go-based architecture: Compiled binary with no runtime dependency vulnerabilities
One-line migration: Drop-in OpenAI-compatible API
Better performance: Claims 50x faster P99 latency
Enterprise features: Built-in load balancing, fallbacks, semantic caching

Solution Overview

Bifrost provides a unified OpenAI-compatible API that works with existing SDKs. The migration requires only a base URL change in most cases.

Here’s the comparison:

Feature	LiteLLM	Bifrost
Language	Python	Go
Latency overhead	Higher (Python runtime)	11 microseconds at 5k RPS
Migration effort	N/A	One-line URL change
Supply chain risk	Higher (PyPI dependencies)	Lower (compiled binary)
License	MIT	Apache 2.0
Provider support	100+ LLMs	15+ providers

Step 1: Install and Run Bifrost

Getting Bifrost running takes about 30 seconds:

# Option A: NPX (fastest)
npx -y @maximhq/bifrost

# Option B: Docker (production)
docker run -p 8080:8080 maximhq/bifrost

# Open Web UI for configuration
open http://localhost:8080

I prefer Docker for production deployments because it isolates the gateway from my host system.

Step 2: Migrate OpenAI SDK

The key change is updating the base_url parameter. Everything else stays the same.

Before (LiteLLM Proxy):

import openai

client = openai.OpenAI(
    api_key="your-litellm-key",
    base_url="http://your-litellm-proxy:4000"
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

After (Bifrost):

import openai

client = openai.OpenAI(
    api_key="your-bifrost-key",
    base_url="http://localhost:8080/openai"  # ONE-LINE CHANGE
)

response = client.chat.completions.create(
    model="gpt-4o-mini",  # Same model name
    messages=[{"role": "user", "content": "Hello!"}]
)

That’s it. One line changed. The model names stay the same, the API calls stay the same.

Step 3: Migrate Anthropic SDK

If you use the Anthropic SDK, the migration is equally simple.

Before (LiteLLM):

from anthropic import Anthropic

client = Anthropic(
    api_key="your-litellm-key",
    base_url="http://your-litellm-proxy:4000/anthropic"
)

After (Bifrost):

from anthropic import Anthropic

client = Anthropic(
    api_key="your-bifrost-key",
    base_url="http://localhost:8080/anthropic"  # ONE-LINE CHANGE
)

Step 4: Configure Providers

Bifrost needs provider configuration before it can route requests. I do this through the Web UI or API.

Via Web UI:

Open http://localhost:8080 and add your provider credentials through the interface.

Via API:

curl -X POST http://localhost:8080/v1/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openai",
    "api_key": "sk-..."
  }'

For Azure OpenAI:

curl -X POST http://localhost:8080/v1/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "azure",
    "api_key": "your-azure-key",
    "api_base": "https://your-azure.openai.azure.com"
  }'

Step 5: Migrate from LiteLLM SDK (Direct Usage)

If you use the LiteLLM Python SDK directly (not through proxy), you need to switch to the OpenAI SDK pattern.

Before (Direct LiteLLM SDK):

from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "your-key"
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

After (Bifrost with OpenAI SDK):

import openai

client = openai.OpenAI(
    api_key="your-bifrost-key",
    base_url="http://localhost:8080/openai"
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

This change requires more code modification, but it standardizes on the OpenAI SDK which is widely supported.

Step 6: Migrate config.yaml

If you use LiteLLM’s config.yaml for provider configuration, you need to migrate those settings to Bifrost.

LiteLLM config.yaml:

model_list:
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4-deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY

Bifrost equivalent (configure via API):

curl -X POST http://localhost:8080/v1/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "azure",
    "api_key": "your-azure-key",
    "api_base": "https://your-azure.openai.azure.com"
  }'

Bifrost doesn’t use a YAML config file by default. I find the Web UI more convenient for managing providers.

Step 7: Test the Migration

Before going to production, I always run a test script:

import openai

# Test with Bifrost
client = openai.OpenAI(
    api_key="test-key",
    base_url="http://localhost:8080/openai"
)

try:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Test migration"}]
    )
    print("Migration successful!")
    print(response.choices[0].message.content)
except Exception as e:
    print(f"Migration failed: {e}")

If this works, the core migration is complete.

Step 8: Verify Security

A Reddit commenter made an important point: “Publishing pipeline is the real evaluation criterion here. Check their .github/workflows before migrating.”

I take this seriously:

git clone https://github.com/maximhq/bifrost
cd bifrost
cat .github/workflows/*.yml

I look for:

Are secrets properly managed?
Is code review required?
Are dependencies pinned?
Are there security scans?

This verification helps me trust the project’s supply chain security.

Common Mistakes

I made these mistakes during my migration:

Not testing performance first: I should have run actual workloads in staging for a few days before full migration. The performance differences only matter if they match my use case.
Assuming feature parity: Bifrost has different advanced features like MCP gateway and semantic caching. I needed to learn these separately.
Skipping provider reconfiguration: I initially assumed Bifrost would automatically pick up environment variables. It requires explicit provider setup.
Not migrating virtual keys: Budget management and access control work differently in Bifrost. I needed to set these up fresh.

Migration Checklist

Before I declare the migration complete, I check:

Summary

Migrating from LiteLLM to Bifrost requires minimal code changes. The key advantages I gained:

Minimal code changes: One-line base URL update for most SDKs
Better security: No Python runtime vulnerabilities
Superior performance: 50x faster P99 latency claims
Drop-in compatibility: Works with existing OpenAI/Anthropic SDKs

The entire migration took me under an hour for a medium-sized application. I recommend running both LiteLLM and Bifrost in staging simultaneously for a few days before full migration. Verify performance gains match your specific workload, and always review the publishing pipeline security before trusting any AI gateway.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!