Text Simulation Integration Guide

This guide explains how to integrate your conversational agent with Evalion's text simulation system for automated testing.

Overview

Evalion tests your agent by simulating realistic conversations. The integration is straightforward:

You provide a webhook endpoint that receives messages
You respond by POSTing back to Evalion's API
The conversation continues until either side ends it

┌─────────────────┐                      ┌─────────────────┐
│                 │  1. POST message     │                 │
│     Evalion     │ ─────────────────▶   │   Your Agent    │
│  (Test Agent)   │                      │   (Endpoint)    │
│                 │  2. POST response    │                 │
│                 │ ◀─────────────────   │                 │
└─────────────────┘                      └─────────────────┘
         │                                        │
         └────────── repeat until end ────────────┘

What You Need to Implement

Your Webhook Endpoint

Provide a single HTTP endpoint that:

Accepts POST requests with JSON body
Processes incoming messages from Evalion
Sends responses back to the reply_to_url

Message Payloads

Messages You'll Receive (Evalion → You)

When Evalion sends a message to your endpoint:

{
    "message": "Hi, I'm having trouble with my account login.",
    "status": "start",
    "simulation_id": "550e8400-e29b-41d4-a716-446655440000__1",
    "message_id": "msg_001",
    "reply_to_url": "https://api.evalion.ai/api/v1/text/simulation"
}

Field	Type	Description
`message`	string	The message content from Evalion's test persona
`status`	string	Conversation state: `"start"`, `"ongoing"`, or `"end"`
`simulation_id`	string	Unique identifier for this conversation (format: `UUID__run_id`)
`message_id`	string	Unique identifier for this specific message
`reply_to_url`	string	The URL where you should POST your response

Messages You Send Back (You → Evalion)

POST your agent's response to the reply_to_url:

{
    "message": "I'd be happy to help with your login issue. Can you tell me what error you're seeing?",
    "status": "ongoing",
    "simulation_id": "550e8400-e29b-41d4-a716-446655440000__1",
    "message_id": "resp_001"
}

Field	Type	Required	Description
`message`	string	✅	Your agent's response
`status`	string	✅	`"start"`, `"ongoing"`, or `"end"`
`simulation_id`	string	✅	Must match the incoming `simulation_id`
`message_id`	string	Optional	Your own message identifier
`metadata`	object	Optional	Any custom data you want to include

Message Status Values

The status field controls conversation flow:

Status	Meaning	When to Use
`start`	First message	Beginning of conversation
`ongoing`	Conversation continues	Normal back-and-forth
`end`	Conversation finished	When your agent completes the interaction

Important: When you receive status: "end", the conversation is over—no response needed.

Conversation Flow Example

Here's a complete conversation flow:

1. EVALION → YOU
   POST https://your-agent.com/webhook
   {
       "message": "Hello, I need help resetting my password",
       "status": "start",
       "simulation_id": "abc123__1",
       "message_id": "msg_001",
       "reply_to_url": "https://api.evalion.ai/api/v1/text/simulation"
   }

2. YOU → EVALION
   POST https://api.evalion.ai/api/v1/text/simulation
   {
       "message": "I can help you reset your password. What email is associated with your account?",
       "status": "ongoing",
       "simulation_id": "abc123__1"
   }

3. EVALION → YOU
   POST https://your-agent.com/webhook
   {
       "message": "It's [email protected]",
       "status": "ongoing",
       "simulation_id": "abc123__1",
       "message_id": "msg_002",
       "reply_to_url": "https://api.evalion.ai/api/v1/text/simulation"
   }

4. YOU → EVALION
   POST https://api.evalion.ai/api/v1/text/simulation
   {
       "message": "I've sent a password reset link to [email protected]. Is there anything else I can help with?",
       "status": "ongoing",
       "simulation_id": "abc123__1"
   }

5. EVALION → YOU
   POST https://your-agent.com/webhook
   {
       "message": "No, that's all. Thank you!",
       "status": "ongoing",
       "simulation_id": "abc123__1",
       "message_id": "msg_003",
       "reply_to_url": "https://api.evalion.ai/api/v1/text/simulation"
   }

6. YOU → EVALION (ending the conversation)
   POST https://api.evalion.ai/api/v1/text/simulation
   {
       "message": "You're welcome! Have a great day!",
       "status": "end",
       "simulation_id": "abc123__1"
   }

## Configuration In the agent configuration, edit mode, you need to specify the endpoint URL for your webhook. This is where Evalion will send messages to your agent during testing.

Navigate to the Agent defined in Evalion platform, and select the "Edit" option. Make sure the Interaction mode is set to Text or Voice and Text depending on your situation. In the "Text Integration Configuration" section, you will find the "Endpoint URL" field. Enter the URL of your webhook endpoint there. You can specify some configuration elements to the text conversation such as: Timeout, Max Messages and Conversation Timeout.

It is very important to set the Headers if your endpoint requires authentication.

That is all

Implementation Example

Python (FastAPI)

from fastapi import FastAPI, BackgroundTasks
import httpx

app = FastAPI()

@app.post("/webhook")
async def receive_message(payload: dict, background_tasks: BackgroundTasks):
    # Don't respond if conversation ended
    if payload.get("status") == "end":
        return {"status": "received"}
    
    # Process in background to respond quickly
    background_tasks.add_task(
        process_and_respond,
        payload["message"],
        payload["simulation_id"],
        payload["reply_to_url"]
    )
    
    return {"status": "received"}

async def process_and_respond(message: str, simulation_id: str, reply_to_url: str):
    # Generate your agent's response
    response_text = await your_agent.generate_response(message)
    
    # Determine if conversation should end
    should_end = your_agent.should_end_conversation()
    
    # Send response back to Evalion
    async with httpx.AsyncClient() as client:
        await client.post(
            reply_to_url,
            json={
                "message": response_text,
                "status": "end" if should_end else "ongoing",
                "simulation_id": simulation_id,
            },
            timeout=30.0,
        )

Node.js (Express)

const express = require('express');
const axios = require('axios');

const app = express();
app.use(express.json());

app.post('/webhook', async (req, res) => {
    const { message, status, simulation_id, reply_to_url } = req.body;
    
    // Acknowledge receipt immediately
    res.json({ status: 'received' });
    
    // Don't respond if conversation ended
    if (status === 'end') return;
    
    // Generate response asynchronously
    const responseText = await yourAgent.generateResponse(message);
    const shouldEnd = yourAgent.shouldEndConversation();
    
    // Send response back to Evalion
    await axios.post(reply_to_url, {
        message: responseText,
        status: shouldEnd ? 'end' : 'ongoing',
        simulation_id: simulation_id,
    });
});

Response Expectations

When Evalion sends you a message, here's what we expect:

Requirement	Details
Acknowledge quickly	Return HTTP 200 within a few seconds
Respond asynchronously	POST your actual response to `reply_to_url`
Response time	Send your response within 30 seconds
Match simulation_id	Always include the same `simulation_id` you received

Expected HTTP Response to Webhooks

Your endpoint should return HTTP 200 with a simple acknowledgment:

{
    "status": "received"
}

Ending Conversations

A conversation ends when:

You end it — Send status: "end" in your response
Evalion ends it — You receive status: "end" (no response needed)
Max messages reached — Conversation auto-terminates after ~50 messages

When you receive a message with status: "end", simply acknowledge it—don't send a response back.

Configuration Checklist

Before testing, ensure you've configured:

Endpoint URL — The webhook URL where Evalion sends messages
Authentication — Any headers needed to access your endpoint (API keys, tokens)
Timeout settings — Your endpoint should handle requests within 30 seconds

Provide these details to Evalion when setting up your agent:

{
    "endpoint": "https://your-agent.com/webhook",
    "headers": {
        "Authorization": "Bearer your-api-key",
        "X-Custom-Header": "optional-value"
    }
}

Troubleshooting

Not receiving messages?

Verify your endpoint is publicly accessible
Check firewall rules allow incoming POST requests
Confirm the endpoint URL is correctly configured in Evalion

Responses not being received by Evalion?

Ensure you're POSTing to the exact reply_to_url provided
Verify simulation_id matches exactly
Check your response includes required fields (message, status, simulation_id)

Conversations ending unexpectedly?

Make sure you're not accidentally sending status: "end" too early
Check for errors in your response generation that might cause timeouts

Quick Reference

Your endpoint receives:

{
    "message": "string",
    "status": "start | ongoing | end",
    "simulation_id": "string",
    "message_id": "string",
    "reply_to_url": "string"
}

You respond with:

{
    "message": "string",
    "status": "ongoing | end",
    "simulation_id": "string"
}

POST responses to: The reply_to_url from the incoming message

Questions? Contact [email protected]