Eval Library
V
For VapiAI Platform

Realtime Voice And Turn Taking

Vapi · Vapi

Voice AI Orchestration — Vapi

Vapi evals — Real-time Voice & Turn-taking (relift v3 InfraRed)

About Vapi

Vapi is a voice-AI orchestration platform that wires speech-to-text, an LLM, and text-to-speech into low-latency phone and web voice agents, with interruption handling, mid-call function calling, transfers, recordings, and telephony routing.

Employees

~50

Industry

Voice AI Orchestration

Headquarters

San Francisco, CA

Website

vapi.ai

Sample tests· showing 3 of 10

#InputExpected behaviorCheck
01

Default startSpeakingPlan.waitSeconds is short. On a noisy call, the assistant starts speaking over the customer's last syllable on most turns.

Raise startSpeakingPlan.waitSeconds (e.g., 0.4 → 0.8) and enable transcriptionEndpointingPlan.smartEndpointingEnabled if available so end-of-turn is detected from transcript shape, not just silence. Validate by measuring interruption rate in artifact.messages[] before/after.

Pass / FailAi Platformhigh
02

Long customer monologue — assistant stays silent for 30 seconds. Customer says 'are you still there?'

Enable backchanneling (mm-hmm / right / I see) on the assistant so the agent vocalizes presence without taking the turn. Configure acknowledgementPhrases or rely on the documented default — verify in artifact.messages[] that backchannel tokens fire mid-monologue, not after.

Pass / FailAi Platformmedium
03

Customer says 'wait —' and the assistant keeps talking for another 4 seconds before stopping. Operator wants snappier interruption.

Lower stopSpeakingPlan.numWords (e.g., 2 → 1) so the assistant aborts TTS after the first user word, and verify stopSpeakingPlan.voiceSeconds and backoffSeconds are low. Test with deliberate barge-ins; measure assistant.speech.interrupted hook firing latency.

Pass / FailAi Platformhigh

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Vapi
  • Ai Platform
  • Realtime Voice And Turn Taking

Recommended for

VapiVapi customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.