How I Built a Voice AI Agent That Handles Missed Calls and Captures Leads 24/7

Mar 13, 2026

Purpose

This post shows how I built a voice AI agent that answers missed calls 24/7, qualifies leads, and logs everything to my CRM.

The Problem

I run a small plumbing business. For three years, I assumed we always called back missed customers. One day I decided to actually track this.

What I found shocked me:

Week 1: 18 missed calls, 7 never called back (39%)
Week 2: 22 missed calls, 9 never called back (41%)
Week 3: 15 missed calls, 6 never called back (40%)

Average job value: $500
Lost revenue per month: ~$12,000

Almost 40% of missed calls never resulted in a callback. These potential customers called someone else who answered.

I also realized something else. I had constant low-level anxiety about missing important calls. Every time my phone rang during dinner or family time, I felt I had to answer.

Why Voicemail Does Not Work

I tried voicemail first. I set up a professional greeting asking callers to leave their info.

Results:

Total missed calls: 50
Left voicemail: 18 (36%)
Left complete info: 7 (14%)
Converted to jobs: 2 (4%)

Most people hang up on voicemail. They want to talk to someone now, not leave a message and wait.

My Solution: Voice AI Agent

I decided to build a voice AI agent that:

Answers calls 24/7
Greets callers professionally
Asks qualifying questions
Captures contact info
Logs everything to my CRM
Sends me alerts for emergencies

Platform Selection

I tested three options:

Option 1: Twilio + OpenAI (What I Chose)

Full control over the conversation flow. More setup work but completely customizable.

Pros: Maximum flexibility, pay-as-you-go pricing, I own the code Cons: More complex setup, I handle the AI integration

Option 2: Vapi

Purpose-built for voice AI. Faster setup but less customization.

Pros: Very fast to implement, handles complexity for you Cons: Less control, monthly platform fees

Option 3: Bland AI

Enterprise-focused with built-in CRM integrations.

Pros: Strong transcription, ready-made integrations Cons: Higher cost, overkill for my needs

I chose Twilio + OpenAI because I wanted full control and lower ongoing costs.

Building the Voice AI Agent

Step 1: Basic Twilio Setup

First, I set up a Twilio phone number to receive calls:

const express = require('express');
const twilio = require('twilio');

const app = express();

app.post('/voice', twilio.webhook(), async (req, res) => {
  const twiml = new twilio.twiml.VoiceResponse();

  // Answer the call
  twiml.say({
    voice: 'Polly.Joanna',
    language: 'en-US'
  }, 'Thank you for calling ABC Plumbing. ' +
     'Our team is currently on another call. ' +
     'I can help you right now and have someone follow up shortly.');

  // Connect to AI for conversation
  const connect = twiml.connect();
  connect.stream({
    url: 'wss://my-server.com/media-stream',
  });

  res.type('text/xml');
  res.send(twiml.toString());
});

app.listen(3000);

This answers the call and connects the audio stream to my AI service.

Step 2: AI Conversation Handler

I created a WebSocket handler for real-time conversation:

const { WebSocketServer } = require('ws');
const { OpenAI } = require('openai');

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const wss = new WebSocketServer({ port: 8080 });

const SYSTEM_PROMPT = `You are a friendly assistant for ABC Plumbing.
Your job is to:
1. Greet the caller warmly
2. Ask what service they need
3. Get their name and phone number
4. Ask if it's an emergency (water leaking, no hot water, etc.)
5. Get their address if possible

Keep responses short and natural. Speak clearly.`;

wss.on('connection', async (ws) => {
  let conversationHistory = [
    { role: 'system', content: SYSTEM_PROMPT }
  ];

  ws.on('message', async (message) => {
    const data = JSON.parse(message);

    if (data.event === 'media') {
      // Decode incoming audio
      const audioBuffer = Buffer.from(data.media.payload, 'base64');

      // Transcribe with Whisper
      const transcription = await openai.audio.transcriptions.create({
        file: audioBuffer,
        model: 'whisper-1',
        language: 'en'
      });

      conversationHistory.push({
        role: 'user',
        content: transcription.text
      });

      // Get AI response
      const response = await openai.chat.completions.create({
        model: 'gpt-4',
        messages: conversationHistory,
        max_tokens: 100
      });

      const aiText = response.choices[0].message.content;
      conversationHistory.push({
        role: 'assistant',
        content: aiText
      });

      // Generate speech (using ElevenLabs or OpenAI TTS)
      const audioResponse = await generateSpeech(aiText);

      // Send audio back to Twilio
      ws.send(JSON.stringify({
        event: 'media',
        media: { payload: audioResponse.toString('base64') }
      }));
    }
  });
});

Step 3: Simplified Version with Vapi

If you want to skip the complex setup, here’s a Vapi version:

const response = await fetch('https://api.vapi.ai/assistant', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.VAPI_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    name: 'ABC Plumbing Assistant',
    model: {
      provider: 'openai',
      model: 'gpt-4',
      messages: [
        {
          role: 'system',
          content: `You are a friendly assistant for ABC Plumbing.
            Ask what service they need, get their name, phone, and address.
            Ask if it's an emergency. Be concise and friendly.`
        }
      ]
    },
    voice: {
      provider: 'eleven-labs',
      voiceId: 'Joanna'
    },
    functions: [
      {
        name: 'create_lead',
        description: 'Create a lead in our CRM',
        parameters: {
          type: 'object',
          properties: {
            name: { type: 'string', description: 'Caller name' },
            phone: { type: 'string', description: 'Phone number' },
            address: { type: 'string', description: 'Service address' },
            service_type: { type: 'string', description: 'Plumbing service needed' },
            is_emergency: { type: 'boolean', description: 'Is this an emergency?' }
          },
          required: ['name', 'phone', 'service_type']
        }
      }
    ]
  })
});

const assistant = await response.json();
console.log('Assistant ID:', assistant.id);

Step 4: CRM Integration

I connected the agent to my CRM (HubSpot in my case):

from flask import Flask, request, jsonify
import hubspot
from hubspot.crm.objects import SimplePublicObjectInput
import os

app = Flask(__name__)
hubspot_client = hubspot.Client.create(
    access_token=os.getenv('HUBSPOT_API_KEY')
)

@app.route('/webhook/lead-captured', methods=['POST'])
def create_lead():
    data = request.json

    # Create contact
    contact = hubspot_client.crm.objects.basic_api.create_object(
        'contacts',
        SimplePublicObjectInput(properties={
            'firstname': data.get('name', '').split()[0],
            'lastname': ' '.join(data.get('name', '').split()[1:]),
            'phone': data.get('phone'),
            'address': data.get('address'),
            'service_needed': data.get('service_type'),
            'lead_source': 'Voice AI - Missed Call',
            'lead_status': 'NEW',
            'priority': 'HIGH' if data.get('is_emergency') else 'MEDIUM'
        })
    )

    # Create follow-up task
    task = hubspot_client.crm.objects.basic_api.create_object(
        'tasks',
        SimplePublicObjectInput(properties={
            'hs_task_subject': f"Follow up with {data.get('name')}",
            'hs_task_body': f"Call from Voice AI\nPhone: {data.get('phone')}\nEmergency: {data.get('is_emergency')}",
            'hs_task_status': 'NOT_STARTED',
            'hs_task_priority': 'HIGH' if data.get('is_emergency') else 'MEDIUM'
        })
    )

    # Send alert for emergencies
    if data.get('is_emergency'):
        send_sms_alert(data)

    return jsonify({
        'success': True,
        'contact_id': contact.id,
        'task_id': task.id
    })

def send_sms_alert(data):
    # Use Twilio to send SMS to my phone
    from twilio.rest import Client
    client = Client(os.getenv('TWILIO_SID'), os.getenv('TWILIO_TOKEN'))
    client.messages.create(
        body=f"EMERGENCY: {data.get('name')} at {data.get('address')}. Call {data.get('phone')}",
        from_=os.getenv('TWILIO_PHONE'),
        to=os.getenv('MY_PHONE')
    )

if __name__ == '__main__':
    app.run(port=5000)

Testing the System

I tested with friends and family before going live:

Test 1: Normal inquiry - Captured name, phone, service type
Test 2: Emergency call - Got alert within 30 seconds
Test 3: Unclear speech - Agent asked caller to repeat
Test 4: Hang up - Logged partial info
Test 5: Wrong number - Agent handled gracefully

The ROI Calculation

I wrote a simple script to calculate the return on investment:

def calculate_missed_call_roi(
    missed_calls_per_week: int,
    percent_never_callback: float,
    avg_job_value: float,
    ai_capture_rate: float = 0.5,
    close_rate: float = 0.6,
    cost_per_minute: float = 0.03,
    avg_call_duration: float = 3
):
    """
    Calculate ROI of voice AI agent.

    Example for my plumbing business:
    - 20 missed calls/week
    - 40% never callback
    - $500 average job value
    """
    weekly_lost_leads = missed_calls_per_week * percent_never_callback
    annual_lost_leads = weekly_lost_leads * 52

    # Revenue lost without AI
    annual_revenue_lost = annual_lost_leads * avg_job_value * close_rate

    # Revenue recovered with AI
    leads_captured = annual_lost_leads * ai_capture_rate
    revenue_recovered = leads_captured * avg_job_value * close_rate

    # AI costs
    weekly_ai_cost = missed_calls_per_week * avg_call_duration * cost_per_minute
    annual_ai_cost = weekly_ai_cost * 52

    net_annual_benefit = revenue_recovered - annual_ai_cost

    return {
        'annual_leads_lost': annual_lost_leads,
        'annual_revenue_lost': annual_revenue_lost,
        'leads_captured_with_ai': leads_captured,
        'revenue_recovered': revenue_recovered,
        'annual_ai_cost': annual_ai_cost,
        'net_annual_benefit': net_annual_benefit
    }

# My calculation
result = calculate_missed_call_roi(
    missed_calls_per_week=20,
    percent_never_callback=0.40,
    avg_job_value=500
)

print(f"Annual revenue lost: ${result['annual_revenue_lost']:,.0f}")
print(f"Revenue recovered with AI: ${result['revenue_recovered']:,.0f}")
print(f"Annual AI cost: ${result['annual_ai_cost']:,.0f}")
print(f"Net benefit: ${result['net_annual_benefit']:,.0f}")

Output:

Annual revenue lost: $124,800
Revenue recovered with AI: $62,400
Annual AI cost: $94
Net benefit: $62,306

The AI pays for itself within the first week.

What I Learned

Mistake 1: Over-engineering the script

My first script had 10 questions. Callers hung up. I simplified to 4 key questions: name, phone, service type, emergency status.

Mistake 2: No escalation path

Some callers insisted on talking to a human. I added: “Would you like me to have someone call you back within 5 minutes?” This option reduced hang-ups.

Mistake 3: Not testing accents

The AI struggled with strong accents. I trained it with more diverse speech samples and added: “Could you spell that for me?” as a fallback.

What worked:

Keep the initial greeting under 10 seconds
Offer a “press 1 for callback” option
Send SMS confirmation to the caller
Review call transcripts weekly to improve the script

Unexpected Benefit

Beyond capturing leads, I noticed something else. The anxiety about missing calls disappeared. I knew every call would be handled professionally. I could focus on the job in front of me without worrying about the phone.

Summary

In this post, I showed how I built a voice AI agent for missed call handling. The key steps were: choosing a platform (Twilio + OpenAI for control, Vapi for simplicity), setting up the call handler, connecting to my CRM, and testing thoroughly.

The system captures 50% of leads that would have been lost, recovering over $60,000 annually for my business. The anxiety of missing calls is gone.

Start by tracking your missed call rate. Then build a simple version and improve it based on real conversations.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Twilio Voice API
👨‍💻 Vapi - Voice AI Platform
👨‍💻 Bland AI
👨‍💻 OpenAI Whisper

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!