VAPI Integration

Integrate your VAPI voice agents with Evalion to monitor and evaluate real conversations in real-time. This guide walks you through the complete setup process.

Prerequisites

Before you begin, ensure you have:

• An active VAPI account with at least one deployed agent
• Admin access to your VAPI dashboard
• An Evalion project where you want to monitor the agent

Setup Steps

Step 1: Get Your VAPI API Key

1. Navigate to the VAPI API Keys page
2. Locate or create a Private API Key
3. Copy the API key (you'll need it in Step 3)

Keep your API key secure and never share it publicly. If you suspect your key has been compromised, regenerate it immediately from the VAPI dashboard.

Step 2: Configure Webhook for Real-time Monitoring

To enable real-time conversation monitoring, you need to configure VAPI to send webhook events to Evalion.

1. In your VAPI dashboard, navigate to your agent's settings
2. Find the Messaging section
3. Locate the Server URL field
4. Add the following webhook URL:

{YOUR_API_URL}/v1/api/integrations/vapi

Replace {YOUR_API_URL} with your Evalion instance URL (e.g., https://api.evalion.ai)

The webhook URL must be accessible from the internet. If you're using a self-hosted Evalion instance, ensure your firewall allows incoming requests from VAPI's servers.

Step 3: Get Your VAPI Agent ID

1. In your VAPI dashboard, navigate to your agent
2. The agent ID is typically visible in the URL or agent details section
3. Copy the agent ID (format: usually a UUID like abc123-def456-...)

Step 4: Create Monitor Agent in Evalion

1. Log in to your Evalion dashboard
2. Navigate to your project
3. Go to the Agents section
4. Click Create Monitor Agent
5. Select VAPI as the platform
6. Fill in the required information:
   • Agent Name: A descriptive name for identification in Evalion
   • API Key: Paste your VAPI Private API Key from Step 1
   • Agent ID: Paste your VAPI Agent ID from Step 3
7. Click Create or Connect

Evalion will validate your credentials and fetch agent details from VAPI.

Step 5: Import Historical Data (Optional)

If you want to analyze past conversations:

1. Navigate to the monitor agent you just created
2. Look for the Import Historical Data option
3. Configure import settings:
   • Start Date: Beginning of the date range
   • End Date: End of the date range
   • Preview Mode: Enable to review data before importing
4. Click Import

Evalion will fetch and process historical calls from VAPI. Large imports may take several minutes.

Historical import respects VAPI's rate limits. If you have a large volume of calls, the import may be processed in batches.

What Data is Imported

When you connect a VAPI agent, Evalion imports the following data for each call:

Conversation Metadata

• Call duration
• Start and end timestamps
• Call status and outcome
• Caller information (if available)

Transcript Data

• Complete conversation transcript
• Speaker labels (user vs. agent)
• Message timestamps
• Turn-by-turn breakdown

Cost Information

• Per-minute cost
• Total call cost
• Cost breakdown by service

Audio & Analysis

• Recording URLs for playback
• Call summary (if generated by VAPI)
• Message-level analysis
• Sentiment and intent data (if available)

Real-time Monitoring

Once the webhook is configured, Evalion will automatically receive and process new conversations:

1. Call Starts: User interacts with your VAPI agent
2. Call Ends: VAPI sends webhook event to Evalion
3. Processing: Evalion receives the conversation data
4. Evaluation: Your configured metrics are applied automatically
5. Visibility: Results appear in your Evalion dashboard

Webhook Event Types

VAPI sends various event types. Evalion processes:

• conversation.ended: Triggered when a call completes
• Additional events may be supported based on VAPI's webhook specifications

Troubleshooting

Webhook Not Receiving Data

Symptoms: No new conversations appear in Evalion after calls are completed

Solutions:

1. Verify the webhook URL is correctly configured in VAPI
2. Ensure your Evalion instance is accessible from the internet
3. Check VAPI's webhook logs for delivery errors
4. Verify the agent is properly configured and active

API Key Authentication Errors

Symptoms: Error message when creating monitor agent or during import

Solutions:

1. Verify you copied the entire API key without extra spaces
2. Ensure you're using a Private API Key, not a public key
3. Check that the API key hasn't been revoked or expired
4. Regenerate the API key in VAPI if necessary

Historical Import Fails

Symptoms: Import process fails or times out

Solutions:

1. Try importing a smaller date range
2. Use preview mode first to verify data access
3. Check VAPI's API status for potential outages
4. Verify your VAPI plan supports API access to historical data

Missing Transcript Data

Symptoms: Calls are imported but transcripts are empty

Solutions:

1. Verify transcription is enabled in your VAPI agent settings
2. Check that the call was long enough to generate a transcript
3. Ensure the call completed successfully (not abandoned or failed)

Limitations

• Read-only Access: Monitor agents cannot modify your VAPI agent's behavior
• Rate Limits: Historical imports are subject to VAPI's API rate limits
• Data Retention: Historical data availability depends on your VAPI plan
• Testing Restriction: Monitor agents cannot be used in Evalion test suites

Data Privacy & Security

• API keys are encrypted at rest in Evalion's database
• Conversation data is stored according to your Evalion plan's data retention policy
• Data transmission uses HTTPS/TLS encryption
• You can revoke access by removing the monitor agent or rotating API keys

Next Steps

• Configure Evaluation Metrics

Additional Resources

• VAPI Documentation
• VAPI Webhooks Guide
• Monitor Agent System Overview