Skip to content
BSWEN
Blog

How to Migrate from OpenAI Chat Completions to Responses API: Step-by-Step Guide

Cowrie
Cowrie
Dev @ Bswen

My OpenAI costs were getting out of hand. Every month, I’d stare at my API usage dashboard, watching the dollars pile up. I was using the Chat Completions API for everything—from simple text generation to multi-turn conversations—but something felt off.

Then I discovered the Responses API.

OpenAI introduced it as a cleaner, more performant alternative to Chat Completions. The documentation promised 40% to 80% cost reduction and better reasoning model performance. That caught my attention.

But migrating production code? That’s scary. I didn’t want to break anything.

So I approached it methodically. This is my journey through the 7-step migration process—what worked, what didn’t, and why you should consider making the switch.

The Problem: Why Migrate at All?

Before diving into the migration steps, let me explain why I even bothered.

I had three main pain points with Chat Completions:

1. Manual Context Management

For multi-turn conversations, I had to manually track and concatenate messages. Here’s what I was doing:

chat-completions-context.js
let messages = [
    { role: 'user', content: 'What is the capital of France?' }
];


const res1 = await client.chat.completions.create({
    model: 'gpt-4',
    messages
});


// Manually append response to context
messages = messages.concat([res1.choices[0].message]);


// Add next user message
messages.push({ role: 'user', content: 'And its population?' });


const res2 = await client.chat.completions.create({
    model: 'gpt-4',
    messages
});

This worked, but it was fragile. One wrong .concat() call and my context was corrupted.

2. Rising Costs

My monthly OpenAI bill was creeping up. Cache utilization wasn’t great with Chat Completions, especially for repeated contexts.

3. Limited Native Tools

Want web search? You need to implement it yourself. File search? Custom code. Code interpreter? Build it from scratch.

The Responses API promised to solve all three. But the migration documentation was dense. I needed a practical, step-by-step approach.

Here’s what I learned.

Step 1: Update Your Endpoint (The Easy Part)

The first change is trivial but fundamental. You’re switching from:

POST /v1/chat/completions

to:

POST /v1/responses

In the SDK, this translates to:

endpoint-change.js
// Old: Chat Completions
const completion = await client.chat.completions.create({
    model: 'gpt-4',
    messages: [
        { role: 'system', content: 'You are helpful.' },
        { role: 'user', content: 'Hello!' }
    ]
});


// New: Responses API
const response = await client.responses.create({
    model: 'gpt-4',
    instructions: 'You are helpful.',
    input: 'Hello!'
});

Notice the structural change? Let’s dive deeper into that.

Step 2: Rethink Your Message Structure

This is where things get interesting. Chat Completions uses a messages array with roles:

old-message-structure.js
messages: [
    { role: 'system', content: '...' },
    { role: 'user', content: '...' },
    { role: 'assistant', content: '...' },
    { role: 'user', content: '...' }
]

Responses API flattens this concept:

  • instructions replaces the system message
  • input handles user messages

Here’s the side-by-side comparison:

message-structure-comparison.js
// Chat Completions
const completion = await client.chat.completions.create({
    model: 'gpt-4',
    messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: 'What is machine learning?' }
    ]
});
console.log(completion.choices[0].message.content);
responses-message-structure.js
// Responses API
const response = await client.responses.create({
    model: 'gpt-4',
    instructions: 'You are a helpful assistant.',
    input: 'What is machine learning?'
});
console.log(response.output_text);

Key observation: The Responses API has a cleaner output_text helper. No more digging into choices[0].message.content.

For complex inputs with multiple items (like mixing text and images), you can still pass structured input:

structured-input.js
const response = await client.responses.create({
    model: 'gpt-4-vision-preview',
    instructions: 'Analyze images and provide insights.',
    input: [
        {
            type: 'message',
            role: 'user',
            content: [
                { type: 'input_text', text: 'What is in this image?' },
                { type: 'input_image', image_url: 'https://example.com/image.jpg' }
            ]
        }
    ]
});

Step 3: Simplify Multi-Turn Conversations

This was the biggest win for me.

Remember my manual context management code? Here’s how it changes:

Old approach (Chat Completions):

multi-turn-old.js
// Step 1: First user message
let messages = [
    { role: 'user', content: 'What is the capital of France?' }
];
const res1 = await client.chat.completions.create({
    model: 'gpt-4',
    messages
});


// Step 2: Manually update context
messages = messages.concat([res1.choices[0].message]);
messages.push({ role: 'user', content: 'And its population?' });


// Step 3: Second call with full context
const res2 = await client.chat.completions.create({
    model: 'gpt-4',
    messages
});

New approach (Responses API):

multi-turn-new.js
// Step 1: First request with store: true
const res1 = await client.responses.create({
    model: 'gpt-4',
    input: 'What is the capital of France?',
    store: true  // Important: enables context storage
});


// Step 2: Just reference the previous response
const res2 = await client.responses.create({
    model: 'gpt-4',
    input: 'And its population?',
    previous_response_id: res1.id,
    store: true
});

The previous_response_id parameter is magical. No more manual context tracking. OpenAI handles it on their side.

Alternative approach: If you want full control, you can still pass the conversation history explicitly:

explicit-context.js
const res1 = await client.responses.create({
    model: 'gpt-4',
    input: 'What is the capital of France?'
});


// Pass output directly to next call
const res2 = await client.responses.create({
    model: 'gpt-4',
    input: [
        { type: 'message', role: 'user', content: 'What is the capital of France?' },
        { type: 'message', role: 'assistant', content: res1.output_text },
        { type: 'message', role: 'user', content: 'And its population?' }
    ]
});

Step 4: Understand Statefulness

This tripped me up initially.

The store: true parameter I used above? It tells OpenAI to store your response for future reference (via previous_response_id).

But there’s a tradeoff:

┌─────────────────────────────────────────────────────────────┐
│                    Statefulness Options                      │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  store: true                                                 │
│  ├─ Pros: Simple context management                          │
│  ├─ Cons: Data stored on OpenAI servers                      │
│  └─ Use when: Building chatbots, multi-turn flows            │
│                                                              │
│  store: false (or omit)                                      │
│  ├─ Pros: No server-side storage                             │
│  ├─ Cons: Manual context management                          │
│  └─ Use when: Privacy-sensitive applications                  │
│                                                              │
│  Zero Data Retention (ZDR):                                  │
│  ├─ Enterprise feature                                        │
│  ├─ Encrypted reasoning (no plaintext storage)               │
│  └─ Consult OpenAI enterprise docs                           │
│                                                              │
└─────────────────────────────────────────────────────────────┘

For my chatbot use case, store: true was perfect. But if you’re handling sensitive data, consider the privacy implications.

Step 5: Update Function Definitions (The Tricky Part)

This one caught me by surprise. Functions in Responses API are strict by default.

Let me show you the difference:

Chat Completions (non-strict):

functions-old.js
const completion = await client.chat.completions.create({
    model: 'gpt-4',
    messages: [
        { role: 'user', content: 'What is the weather in Paris?' }
    ],
    tools: [{
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get current weather',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'City name'
                    }
                },
                required: ['location']
            }
        }
    }]
});

Responses API (strict by default):

functions-new.js
const response = await client.responses.create({
    model: 'gpt-4',
    input: 'What is the weather in Paris?',
    tools: [{
        type: 'function',
        name: 'get_weather',
        description: 'Get current weather',
        parameters: {
            type: 'object',
            properties: {
                location: {
                    type: 'string',
                    description: 'City name'
                }
            },
            required: ['location'],
            additionalProperties: false  // Required for strict mode
        },
        strict: true  // Explicit, but this is the default
    }]
});

Key differences:

  1. Internally-tagged format: Tools are at the top level, not nested under function
  2. Strict mode: Parameters must exactly match the schema
  3. additionalProperties: false: Required in strict mode

I hit validation errors initially because my schemas weren’t strict-compliant. Here’s what I fixed:

strict-schema-fix.js
// ❌ This will fail in strict mode
parameters: {
    type: 'object',
    properties: { ... }
    // Missing: required, additionalProperties
}


// ✅ This works
parameters: {
    type: 'object',
    properties: { ... },
    required: ['field1', 'field2'],  // Must specify all required fields
    additionalProperties: false      // Must be false in strict mode
}

Step 6: Update Structured Outputs

If you’re using response_format in Chat Completions, you need to migrate to text.format:

Old way:

structured-output-old.js
const completion = await client.chat.completions.create({
    model: 'gpt-4',
    messages: [
        { role: 'user', content: 'List 3 fruits' }
    ],
    response_format: {
        type: 'json_schema',
        json_schema: {
            name: 'fruit_list',
            schema: {
                type: 'object',
                properties: {
                    fruits: {
                        type: 'array',
                        items: { type: 'string' }
                    }
                }
            }
        }
    }
});

New way:

structured-output-new.js
const response = await client.responses.create({
    model: 'gpt-4',
    input: 'List 3 fruits',
    text: {
        format: {
            type: 'json_schema',
            name: 'fruit_list',
            schema: {
                type: 'object',
                properties: {
                    fruits: {
                        type: 'array',
                        items: { type: 'string' }
                    }
                },
                required: ['fruits'],
                additionalProperties: false
            },
            strict: true
        }
    }
});

The structure change is straightforward, but don’t forget the strict schema requirements!

Step 7: Embrace Native Tools (Optional but Powerful)

This is where Responses API shines.

Instead of building your own web search integration, you can use the native web_search tool:

web-search-native.js
const response = await client.responses.create({
    model: 'gpt-4-turbo-preview',
    input: 'What are the latest developments in quantum computing in 2024?',
    tools: [{
        type: 'web_search'
    }]
});

That’s it. No external APIs, no custom implementations.

Other native tools:

File Search:

file-search.js
const response = await client.responses.create({
    model: 'gpt-4-turbo-preview',
    input: 'Find documents about revenue projections',
    tools: [{
        type: 'file_search'
    }]
});

Code Interpreter:

code-interpreter.js
const response = await client.responses.create({
    model: 'gpt-4-turbo-preview',
    input: 'Calculate the compound interest on $10,000 at 7% for 10 years',
    tools: [{
        type: 'code_interpreter'
    }]
});

These tools would have required significant custom development with Chat Completions.

The Results: Was It Worth It?

After completing the migration, I tracked my metrics for two weeks:

┌──────────────────────────────────────────────────────────────┐
│                    Migration Results                          │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  Cost Reduction:        62% (measured: $847 → $321/month)     │
│  Latency:               Slightly improved (~8% faster)         │
│  Code Simplification:   Removed ~200 lines of context logic   │
│  Maintainability:      Easier (no manual context tracking)   │
│                                                               │
│  Performance Claim Check:                                     │
│  ├─ SWE-bench improvement: 3% (per OpenAI docs)              │
│  └─ My subjective experience: Noticeable in reasoning tasks  │
│                                                               │
└──────────────────────────────────────────────────────────────┘

The cost savings exceeded the advertised range. Why? Better cache utilization combined with simpler prompts (no need to repeat context).

Common Pitfalls I Encountered

1. Forgetting Strict Mode in Functions

I initially kept my old function schemas. They worked in Chat Completions but threw validation errors in Responses API.

Fix: Add required array and additionalProperties: false to all schemas.

2. Missing output_text Helper

Old habit: I kept using response.choices[0].message.content.

Fix: Use response.output_text for cleaner code.

3. Not Storing Responses for Multi-Turn

I forgot store: true on my first call, then couldn’t use previous_response_id.

Fix: Always include store: true when you plan to reference the response later.

4. Ignoring Native Tools

I spent a week building custom web search integration before realizing it was built-in.

Fix: Check the native tools first—they’re optimized and well-integrated.

Migration Strategy: Incremental Approach

You don’t have to migrate everything at once. Here’s how I did it:

  1. Week 1: Migrated simple, single-turn endpoints (low risk)
  2. Week 2: Migrated multi-turn conversations with previous_response_id
  3. Week 3: Added native tools (web search, file search)
  4. Week 4: Migrated function-calling endpoints (most complex)

This phased approach let me validate each change before moving forward.

Comparison: Chat Completions vs Responses API

Here’s a quick reference table I created during my migration:

┌─────────────────────┬──────────────────────┬─────────────────────┐
│ Feature              │ Chat Completions      │ Responses API        │
├─────────────────────┼──────────────────────┼─────────────────────┤
│ Endpoint             │ /v1/chat/completions │ /v1/responses       │
│ System Instructions  │ messages[role=system]│ instructions param  │
│ User Input           │ messages[role=user]  │ input param         │
│ Output Access        │ choices[0].message    │ output_text helper  │
│ Context Management   │ Manual array concat   │ previous_response_id│
│ Functions            │ Non-strict default    │ Strict by default   │
│ Native Tools         │ None                  │ web_search, etc.    │
│ Cost Efficiency      │ Baseline              │ 40-80% savings      │
│ Future Support       │ Yes (maintained)      │ Primary focus       │
└─────────────────────┴──────────────────────┴─────────────────────┘

Should You Migrate?

Yes, if:

  • You want lower costs (40-80% savings is significant)
  • You use multi-turn conversations (context management is much simpler)
  • You need native tools (web search, file search, code interpreter)
  • You’re building for the future (Responses API is OpenAI’s focus)

Maybe not yet, if:

  • Your codebase is stable and you’re risk-averse
  • You don’t need any new features
  • You’re using an older model that might not support Responses API fully

Remember: Chat Completions API is still supported. You can migrate incrementally.

Final Thoughts

The migration took me about 3 weeks of careful work. Most changes were straightforward—the endpoint switch, message restructuring, even function updates. The only tricky part was understanding strict schemas and the statefulness tradeoffs.

But the payoff was worth it. My code is cleaner, my costs are down, and I have access to powerful native tools I didn’t have before.

If you’re on Chat Completions and looking to optimize, start with a small endpoint. Test the waters. You’ll likely find, as I did, that the Responses API is a significant step forward.

The 7-step migration process is straightforward:

  1. Update endpoints
  2. Restructure messages (instructions + input)
  3. Simplify multi-turn with previous_response_id
  4. Configure statefulness
  5. Update function definitions (strict mode)
  6. Migrate structured outputs
  7. Add native tools as needed

Start today. Your future self (and your budget) will thank you.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

References

  1. OpenAI Responses API Documentation - Official API reference
  2. Migrating from Chat Completions to Responses API - OpenAI’s migration guide

Comments

Tags: