Realtime Voice And Turn Taking
Vapi · Vapi
Voice AI Orchestration — Vapi
Vapi evals — Real-time Voice & Turn-taking (relift v3 InfraRed)
About Vapi
Vapi is a voice-AI orchestration platform that wires speech-to-text, an LLM, and text-to-speech into low-latency phone and web voice agents, with interruption handling, mid-call function calling, transfers, recordings, and telephony routing.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Default startSpeakingPlan.waitSeconds is short. On a noisy call, the assistant starts speaking over the customer's last syllable on most turns. | Raise startSpeakingPlan.waitSeconds (e.g., 0.4 → 0.8) and enable transcriptionEndpointingPlan.smartEndpointingEnabled if available so end-of-turn is detected from transcript shape, not just silence. Validate by measuring interruption rate in artifact.messages[] before/after. | Pass / FailAi Platformhigh |
| 02 | Long customer monologue — assistant stays silent for 30 seconds. Customer says 'are you still there?' | Enable backchanneling (mm-hmm / right / I see) on the assistant so the agent vocalizes presence without taking the turn. Configure acknowledgementPhrases or rely on the documented default — verify in artifact.messages[] that backchannel tokens fire mid-monologue, not after. | Pass / FailAi Platformmedium |
| 03 | Customer says 'wait —' and the assistant keeps talking for another 4 seconds before stopping. Operator wants snappier interruption. | Lower stopSpeakingPlan.numWords (e.g., 2 → 1) so the assistant aborts TTS after the first user word, and verify stopSpeakingPlan.voiceSeconds and backoffSeconds are low. Test with deliberate barge-ins; measure assistant.speech.interrupted hook firing latency. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Vapi
- Ai Platform
- Realtime Voice And Turn Taking
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.