Eval directory · Voice AI

Evals for Verloop.io

Eval coverage for Verloop.io, mapped from its public product surface.

About Verloop.io

Verloop.io offers Gen AI–powered Voice AI Agents that hold real-time, human-like voice conversations for outbound outreach and inbound customer support. The platform covers agent building (recipe builder, voice configuration, pronunciation), telephony integration including custom SIP/BYOC, and post-call insights, with developer APIs for outreach and consumption. Docs also describe multi-language and dialect handling plus testing best practices for production readiness.

Industry

voice AI agent platform for customer support and outbound calling

Website

docs.verloop.io

Use the eval library for Verloop.io

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

Related in Voice AI

All evals →

More Voice AI eval libraries

ElevenLabs7 evals

Coverage map

What would you measure for Verloop.io?

6 scoring areas · 21 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Recipe Builder & Agent Configuration

The documented authoring surface for voice agents: the recipe builder, its blocks, and agent-level settings that shape behavior before a call is placed.

“Voice AI agents are Gen AI powered Voice Agents with low-latency and high accuracy speech recognition” docs.verloop.io

Mapped capabilities

4 capabilities

Recipe builder fundamentals
Explaining the builder's purpose and how a recipe is assembled from blocks.
Recipe block behavior
Distinguishing block types documented under recipe-block, including the API block.
Agent settings
Agent-level configuration options described on the agent-settings page.
Pre-production testing
Applying the documented testing guide to check an agent is production-ready.

Voice Selection & Pronunciation

Choosing and customizing how the agent sounds, per the configure-voices section: stock voice choice, custom voices, and pronunciation overrides.

“Configure your Voice Agent to handle global linguistic nuances, from Khaleeji Arabic to Hinglish and LATAM Spanish.” docs.verloop.io

Mapped capabilities

3 capabilities

Choosing an agent voice
Guidance on selecting a voice for an agent from documented options.
Adding a custom voice
The documented path for bringing a custom voice into the platform.
Pronunciation overrides
Correcting brand names, acronyms, and domain terms via add-pronunciation.

Illustrative example

Input: Our agent keeps mispronouncing our company name, 'Verloop', on live calls. What's the right way in the platform to make it say the name correctly?
Expected behavior: Points to the add-pronunciation capability in the voice configuration section as the documented fix, rather than suggesting the user rewrite the prompt or swap voices. Does not invent markup syntax or field names that the docs do not show.

Telephony Integration & Connectivity

Connecting agents to the phone network: telephony fundamentals, number management, Verloop-provided vs. bring-your-own carrier, and voice setup.

“Our Voice AI Agents can dial to millions of customers and help you scale like never before.” docs.verloop.io

Mapped capabilities

4 capabilities

Telephony fundamentals
Core principles and integration techniques described in the basics page.
Phone number management
Provisioning numbers via Verloop telephony or an integrated provider.
Custom SIP / BYOC
The documented bring-your-own-carrier SIP integration path.
Voice setup & recognition tuning
Voice setup plus the documented speech recognition optimization levers.

Illustrative example

Input: We already have a carrier contract and want to keep our own trunks instead of buying numbers from you. Is that supported, and what is the documented path?
Expected behavior: Confirms bring-your-own-carrier is supported and identifies custom SIP integration (BYOC) as the documented route, contrasting it with using Verloop-provided telephony. Any trunk credentials, IP allowlists, or codec specifics not present in the docs are flagged as needing verification.

Multi-Language, Dialect & Accent Handling

Configuring agents for global markets, including the dialect and accent guidance covering Khaleeji Arabic, Hinglish, and LATAM Spanish.

“Multi-Language Support: Communicate effectively across global markets with support for multiple languages and dialects.” docs.verloop.io

Mapped capabilities

3 capabilities

Multi-language configuration
Setting up an agent to converse across the documented supported languages.
Dialect and accent tuning
Applying the handling-dialects-and-accents guidance to a target market.
Code-mixed speech
Handling mixed-language input such as Hinglish as described in the docs.

Post-Call Insights & Data Consumption

What the platform captures after a call, how insights are set up, and how downstream systems consume that data accurately.

Mapped capabilities

4 capabilities

Insights overview
What post-call insights are and what they report.
Insights setup
Configuring insights per the documented setup page.
Consuming call data
Retrieving and using post-call data in downstream systems.
Insight accuracy practices
Optimization tips from the accurate-voice-agent-insights guide.

Outreach & Consumption APIs

The developer API surface introduced under api-v1 for driving outbound outreach and consuming results programmatically. Public endpoint reference pages are thin in the current docs, so coverage stays at the level the docs actually support.

Mapped capabilities

3 capabilities

API introduction & scope
What the Voice AI Agent APIs cover for outreach and consumption.
Outreach initiation
Programmatically triggering outbound calls as described in the API introduction.
Documentation gaps
Declining to invent endpoint contracts the published reference does not specify.

Coverage is mapped from Verloop.io's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Verloop.io test?+

The coverage map is generated from Verloop.io's own public product surface (voice AI agent platform for customer support and outbound calling): 6 scoring areas — Recipe Builder & Agent Configuration, Voice Selection & Pronunciation, and Telephony Integration & Connectivity, and more — spanning 21 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Verloop.io evals scored?+

Every case generated for Verloop.io — across Recipe Builder & Agent Configuration and Voice Selection & Pronunciation and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Verloop.io library include?+

The full Verloop.io library is built on request. The coverage map spans 6 areas and 21 capabilities (for example, Recipe builder fundamentals and Recipe block behavior under Recipe Builder & Agent Configuration); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Verloop.io or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Verloop.io areas and set them up in a Corsac workspace, where you can run every test case against Verloop.io or your own agent with your own data.