Retell Agent Config And Llm
Retell AI · Retell AI
Voice AI Agents — Retell AI
Retell AI evals — Agent Configuration & LLM (relift v3 InfraRed)
About Retell AI
Retell AI is a platform for building production phone-call voice agents — pairing a conversation engine with telephony, low-latency turn-taking, interruption handling, mid-call functions, post-call analysis, and batch outbound dialing.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator creates an agent and must choose response_engine. They set response_engine.type='retell-llm' but also supply an llm_websocket_url (which belongs to custom-llm). | response_engine.type selects exactly one engine: 'retell-llm' (managed, references a retell_llm_id), 'custom-llm' (your llm_websocket_url), or 'conversation-flow' (a conversation_flow_id). Pick one and supply only its fields — llm_websocket_url is ignored/invalid under retell-llm. Do not mix. | Pass / FailAi Platformhigh |
| 02 | Operator wants the user (caller) to speak first, so the agent stays silent until spoken to. They are unsure whether to omit begin_message or set it to an empty string. | Set begin_message to an empty string to make the agent wait for the user to speak first; a non-empty begin_message makes the agent open the call. Decide deliberately per inbound/outbound direction and verify with a test call which party speaks first. | Pass / FailAi Platformmedium |
| 03 | Integrator wants a managed Retell LLM agent. They call Create Agent first with response_engine referencing an llm_id that does not exist yet. | Create the Retell LLM first (Create Retell LLM → returns llm_id), then create the agent with response_engine={type:'retell-llm', llm_id:<id>}. general_prompt, begin_message, general_tools and model live on the Retell LLM object, not the agent. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Retell
- Ai Platform
- Agent Config And Llm
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.