Eval directory · Customer Support
Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Sierra AI AI products.
Sierra AI builds conversational AI agents for customer experience, designed to handle the full resolution lifecycle across every channel — chat, voice, and messaging. Sierra agents are deployed by leading consumer brands to reduce handle time and improve CSAT.
Use the eval library for Sierra AI
All 70 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.
Related in Customer Support
All evals →Coverage map
7 areas · 70 graded scenarios
Every eval set is graded on
Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls
01
Evaluates Sierra's Brand Voice & Policy Guardrails across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
10 scenarios
02
Evaluates Sierra's Connected Systems & Tool Audit across 11 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
11 scenarios
03
Evaluates Sierra's Escalation & Live Assist Handoff across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
10 scenarios
04
Evaluates Sierra's Experiments & Observability Safety across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
9 scenarios
05
Evaluates Sierra's Knowledge Grounding & Agent Memory across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
10 scenarios
06
Evaluates Sierra's Skills & Intent Routing across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
10 scenarios
07
Evaluates Sierra's Voice Channel & ASR Safety across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Enterprise conversational AI agents eval coverage.
Mapped capabilities
10 scenarios
Each eval pack tests Sierra AI's public product surface — including Brand Voice Policy Guardrails, Connected Systems Tool Audit, Escalation Live Assist Handoff — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.
Every eval set is graded the same way: pass/fail checks plus an LLM judge scoring 1–5 against each case's expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.
The Sierra AI eval library includes 70 graded test cases across 7 eval packs. Each case defines an input, expected behavior, and pass/fail criteria.
Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run every test case against Sierra AI or your own agent with your own data.
