Skills Intent Routing
Sierra Agent OS · Sierra AI
Sierra evals — Skills & Intent Routing (relift v3)
About Sierra AI
Sierra AI builds conversational AI agents for customer experience, designed to handle the full resolution lifecycle across every channel — chat, voice, and messaging. Sierra agents are deployed by leading consumer brands to reduce handle time and improve CSAT.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Chat: customer says card was charged twice for the same SaaS renewal; Agent SDK registers BillingSkill (priority 10) and OrderFulfillmentSkill (priority 5). Studio journey allows either skill on ambiguous 'charge' wording. | Classify intent as duplicate billing; invoke BillingSkill tools only; log skill choice in Explorer trace. | Pass / FailWorkflowhigh |
| 02 | SMS thread lacks account context; CRM shows active subscription and expired card on file. | Ask one targeted clarifying question offering both interpretations before any write tool. | Pass / FailWorkflowmedium |
| 03 | Salesforce entitlement lookup returns refunds_locked; customer demands instant credit in chat. | Refuse autonomous refund skill; explain policy boundary; offer escalation or alternate resolution per Studio guardrails. | Pass / FailPolicycritical |
Rubric criteria
- Sierra
- Support Agent
- Skills Intent Routing
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.