Function Calling and Structured Outputs in Responses API: Key Differences from Chat Completions
I copied my function calling code from Chat Completions to Responses API, ran it, and got this error:
Error: Invalid function definition: 'name' must be at top level, not nested under 'function'My code had worked perfectly in Chat Completions. Why was it suddenly invalid?
Then I tried structured outputs with response_format—the parameter I’d used for years. The API silently ignored it and returned unstructured text.
Two hours later, I finally understood: Responses API changed three fundamental things about function calling and structured outputs. These weren’t optional changes—they were breaking changes that caught me completely off guard.
This is what I learned through trial and error.
The Problem: Three Breaking Changes I Didn’t Expect
When I migrated to Responses API, I assumed function calling would work the same way. It didn’t.
Here’s what broke:
┌─────────────────────────────────────────────────────────────────┐│ Breaking Changes: Chat Completions → Responses │├─────────────────────────────────────────────────────────────────┤│ ││ 1. Function Definition Format ││ Old: Externally tagged (nested under "function") ││ New: Internally tagged (name, description at top level) ││ → My nested schemas threw validation errors ││ ││ 2. Strict Mode ││ Old: Functions non-strict by default ││ New: Functions strict by default ││ → My lenient schemas failed validation ││ ││ 3. Structured Outputs ││ Old: response_format parameter ││ New: text.format object ││ → response_format was silently ignored ││ │└─────────────────────────────────────────────────────────────────┘Each one cost me debugging time. Let me show you exactly what happened and how I fixed it.
Breaking Change 1: Internally-Tagged Function Definitions
I started with a simple weather function that worked in Chat Completions:
// Chat Completions: This worked perfectlyconst completion = await client.chat.completions.create({ model: 'gpt-4', messages: [ { role: 'user', content: 'What is the weather in Paris?' } ], tools: [{ type: 'function', function: { // NESTED under "function" name: 'get_weather', description: 'Get current weather for a location', parameters: { type: 'object', properties: { location: { type: 'string', description: 'City name' } }, required: ['location'] } } }]});I copied this to Responses API and got:
Error 400: Invalid tool definitionDetails: 'name' should be at top level of tool object, not under 'function' keyThe issue: Chat Completions uses “externally tagged” polymorphism—properties nested under a type-specific key (function). Responses API uses “internally tagged” polymorphism—properties at the top level.
Here’s the fix:
// Responses API: Flatten the structureconst response = await client.responses.create({ model: 'gpt-4', input: 'What is the weather in Paris?', tools: [{ type: 'function', name: 'get_weather', // TOP level now description: 'Get current weather for a location', // TOP level parameters: { // TOP level type: 'object', properties: { location: { type: 'string', description: 'City name' } }, required: ['location'], additionalProperties: false // NEW requirement (see below) } }]});Visual comparison:
┌────────────────────────────────────────────────────────────────┐│ Externally vs Internally Tagged │├────────────────────────────────────────────────────────────────┤│ ││ CHAT COMPLETIONS (externally tagged): ││ ┌─────────────────────────────────────────────┐ ││ │ { │ ││ │ "type": "function", │ ││ │ "function": { ← wrapper │ ││ │ "name": "get_weather", ← nested │ ││ │ "description": "...", ← nested │ ││ │ "parameters": { ... } ← nested │ ││ │ } │ ││ │ } │ ││ └─────────────────────────────────────────────┘ ││ ││ RESPONSES API (internally tagged): ││ ┌─────────────────────────────────────────────┐ ││ │ { │ ││ │ "type": "function", │ ││ │ "name": "get_weather", ← top level │ ││ │ "description": "...", ← top level │ ││ │ "parameters": { ... } ← top level │ ││ │ } │ ││ └─────────────────────────────────────────────┘ ││ ││ Key: Remove the "function" wrapper key ││ │└────────────────────────────────────────────────────────────────┘I had 12 function definitions in my codebase. I had to update all of them manually. There’s no automatic conversion.
Breaking Change 2: Strict Mode by Default
After fixing the structure, I thought I was done. But then:
Error 400: Schema validation failedDetails: 'additionalProperties' must be false for strict modeMissing required fields in schema: ['location']My schema had worked fine in Chat Completions because it was non-strict. The model could infer things. But Responses API is strict by default.
Here’s what I had (non-strict, worked in Chat Completions):
// This was fine in Chat Completions (non-strict default)parameters: { type: 'object', properties: { location: { type: 'string' } }, required: ['location'] // additionalProperties not specified - model was lenient}Here’s what Responses API requires (strict default):
// Responses API requires this (strict by default)parameters: { type: 'object', properties: { location: { type: 'string', minLength: 1, // Can add constraints description: 'City name' } }, required: ['location'], additionalProperties: false // MANDATORY in strict mode}Why strict mode matters:
┌──────────────────────────────────────────────────────────────────┐│ Strict vs Non-Strict │├──────────────────────────────────────────────────────────────────┤│ ││ NON-STRICT (Chat Completions default): ││ ├─ Model can infer missing fields ││ ├─ Additional properties allowed ││ ├─ Flexible but unpredictable ││ └─ Works with partial schemas ││ ││ STRICT (Responses API default): ││ ├─ Schema must be complete and exact ││ ├─ additionalProperties: false REQUIRED ││ ├─ All required fields must be in schema ││ ├─ No extra properties allowed ││ └─ Predictable but requires careful schema design ││ ││ Migration Impact: ││ If you relied on lenient parsing, you'll get validation errors ││ │└──────────────────────────────────────────────────────────────────┘Option 1: Make your schemas strict-compliant
This is the recommended approach. Strict schemas are more predictable:
// Complete strict schemaparameters: { type: 'object', properties: { location: { type: 'string', minLength: 1, maxLength: 100, description: 'City name' }, unit: { type: 'string', enum: ['celsius', 'fahrenheit'], description: 'Temperature unit' } }, required: ['location', 'unit'], additionalProperties: false // Always required}Option 2: Explicitly disable strict mode
If you need flexibility, you can set strict: false:
// Explicitly non-strict (but not recommended)tools: [{ type: 'function', name: 'get_weather', description: 'Get weather', parameters: { type: 'object', properties: { location: { type: 'string' } }, required: ['location'] // additionalProperties not needed }, strict: false // EXPLICIT override}]But OpenAI recommends strict mode for better reliability. I chose to update all my schemas.
Breaking Change 3: Structured Outputs Use text.format
I had code using response_format for JSON output:
// Chat Completions: response_format parameterconst completion = await client.chat.completions.create({ model: 'gpt-4', messages: [ { role: 'user', content: 'Jane, 54 years old' } ], response_format: { type: 'json_schema', json_schema: { name: 'person', strict: true, schema: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'number' } }, required: ['name', 'age'], additionalProperties: false } } }});I copied this to Responses API. The call succeeded, but the output was plain text—not JSON.
The problem: Responses API moved structured output configuration from response_format to text.format:
// Responses API: text.format objectconst response = await client.responses.create({ model: 'gpt-4', input: 'Jane, 54 years old', text: { format: { type: 'json_schema', name: 'person', strict: true, schema: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'number' } }, required: ['name', 'age'], additionalProperties: false } } }});
const person = JSON.parse(response.output_text);console.log(person); // { name: 'Jane', age: 54 }Key structural difference:
┌────────────────────────────────────────────────────────────────┐│ Structured Outputs: Parameter vs Object │├────────────────────────────────────────────────────────────────┤│ ││ CHAT COMPLETIONS: ││ response_format: { ← top-level parameter ││ type: 'json_schema', ││ json_schema: { ││ name: 'person', ││ schema: { ... } ││ } ││ } ││ ││ RESPONSES API: ││ text: { ← wrapper object ││ format: { ← nested under 'text' ││ type: 'json_schema', ││ name: 'person', ← no 'json_schema' wrapper ││ schema: { ... } ││ } ││ } ││ ││ Changes: ││ 1. response_format → text.format ││ 2. No 'json_schema' nesting ││ 3. name, strict, schema all at format level ││ │└────────────────────────────────────────────────────────────────┘What confused me: Using response_format in Responses API doesn’t error—it’s just ignored. You get unstructured text back.
The call_id Correlation Pattern
One more thing that tripped me up: handling tool outputs.
In Chat Completions, I used the tool call ID from the response to correlate with my tool output. The pattern in Responses API is similar but the structure is different.
Chat Completions tool handling:
// Chat Completionsconst completion = await client.chat.completions.create({ model: 'gpt-4', messages: [{ role: 'user', content: 'Weather in Paris?' }], tools: [...]});
const toolCall = completion.choices[0].message.tool_calls[0];
// Send tool output backconst result = await client.chat.completions.create({ model: 'gpt-4', messages: [ { role: 'user', content: 'Weather in Paris?' }, completion.choices[0].message, // Include tool call message { role: 'tool', tool_call_id: toolCall.id, // Correlate by ID content: 'Paris: 22C, sunny' } ]});Responses API tool handling:
// Responses APIconst response = await client.responses.create({ model: 'gpt-4', input: 'Weather in Paris?', tools: [...]});
// Find the function_call itemconst functionCall = response.output.find( item => item.type === 'function_call');
// Send tool output backconst result = await client.responses.create({ model: 'gpt-4', previous_response_id: response.id, input: [{ type: 'function_call_output', call_id: functionCall.call_id, // Correlate by call_id output: 'Paris: 22C, sunny' }]});Key difference: Tool calls and outputs are distinct “Item” types, correlated by call_id:
┌──────────────────────────────────────────────────────────────────┐│ Tool Output Pattern: call_id │├──────────────────────────────────────────────────────────────────┤│ ││ CHAT COMPLETIONS: ││ ├─ Tool call in: message.tool_calls[0].id ││ ├─ Tool output: role='tool', tool_call_id ││ └─ Both in messages array ││ ││ RESPONSES API: ││ ├─ Tool call item: type='function_call', has call_id ││ ├─ Tool output item: type='function_call_output', same call_id ││ ├─ Items in output array / input array ││ └─ Can use previous_response_id for context ││ │└──────────────────────────────────────────────────────────────────┘Common Mistakes I Made (And How to Fix Them)
Mistake 1: Copy-pasting function definitions without flattening
Error: 'name' must be at top levelFix: Remove the function: wrapper and move all properties up.
Mistake 2: Not adding additionalProperties: false
Error: Schema validation failed for strict modeFix: Add additionalProperties: false to all object schemas.
Mistake 3: Using response_format in Responses API
No error, but output is unstructured textFix: Use text.format object instead.
Mistake 4: Not understanding call_id correlation
Tool outputs not properly linked to tool callsFix: Use type: 'function_call_output' with matching call_id.
Mistake 5: Assuming non-strict mode still applies
Validation errors on previously working schemasFix: Either make schemas strict-compliant or set strict: false explicitly.
A Checklist for Migration
Before migrating function calling code:
┌────────────────────────────────────────────────────────────────┐│ Function Calling Migration Checklist │├────────────────────────────────────────────────────────────────┤│ ││ [ ] Flatten function definitions ││ - Remove 'function' wrapper ││ - Move name, description, parameters to top level ││ ││ [ ] Update schemas for strict mode ││ - Add additionalProperties: false ││ - Verify all required fields listed ││ - Add constraints if needed (minLength, enum, etc.) ││ ││ [ ] Migrate structured outputs ││ - Change response_format → text.format ││ - Remove json_schema nesting ││ - Move name, strict, schema to format level ││ ││ [ ] Update tool output handling ││ - Use function_call_output type ││ - Correlate with call_id from function_call item ││ - Consider previous_response_id for context ││ ││ [ ] Test all function-calling endpoints ││ - Verify strict schemas pass validation ││ - Confirm tool outputs properly linked ││ - Check structured output format ││ │└────────────────────────────────────────────────────────────────┘Summary: What Changed and Why
┌─────────────────────────────────────────────────────────────────┐│ Migration Summary: Function Calling │├─────────────────────────────────────────────────────────────────┤│ ││ Function Definitions: ││ ├─ From: externally tagged (nested under "function") ││ ├─ To: internally tagged (properties at top level) ││ └─ Reason: Cleaner schema, clearer structure ││ ││ Strict Mode: ││ ├─ From: non-strict by default (lenient validation) ││ ├─ To: strict by default (exact validation) ││ ├─ Requires: additionalProperties: false ││ └─ Reason: Predictable outputs, catches schema errors early ││ ││ Structured Outputs: ││ ├─ From: response_format parameter ││ ├─ To: text.format object ││ ├─ Structure: no json_schema wrapper ││ └─ Reason: Consistent with text configuration approach ││ ││ Tool Outputs: ││ ├─ Correlation: call_id (not tool_call_id) ││ ├─ Item types: function_call, function_call_output ││ └─ Reason: Cleaner item-based architecture ││ │└─────────────────────────────────────────────────────────────────┘The internally-tagged format is objectively cleaner. Strict validation catches problems early. But if you’re migrating existing code, you need to update every function definition and schema.
Should You Migrate Function Calling?
Yes, if:
- You want predictable outputs (strict schemas)
- You’re starting fresh (can design schemas correctly from the start)
- You need cleaner architecture (internally-tagged is simpler)
Maybe wait, if:
- You have many existing function definitions (migration effort)
- You relied on non-strict flexibility (schema updates needed)
- Your codebase is stable and risk-averse
The migration isn’t technically difficult—flatten definitions, add strict requirements, update structured outputs. But it’s tedious if you have many functions.
How I Approached It
I had 12 functions across 4 endpoints. My approach:
- First: Migrated simple functions with basic schemas (2 functions)
- Then: Updated schemas to be strict-compliant (added required fields, additionalProperties: false)
- Finally: Migrated complex function-calling chains with tool outputs
Each endpoint took about 30 minutes. Testing was crucial—I ran every function call through validation to catch schema issues.
The strict schemas actually helped me find bugs in my original code. One function was missing a required field that the model had been inferring. With strict mode, I had to be explicit.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
References
- OpenAI Responses API Function Calling - Official function calling reference
- Migrating from Chat Completions to Responses API - OpenAI’s migration guide
- Structured Outputs Guide - Detailed structured output documentation
Comments