Sierra AI
Eval Factory Imported Suite
About Sierra AI
Sierra AI builds conversational AI agents for customer experience, designed to handle the full resolution lifecycle across every channel — chat, voice, and messaging. Sierra agents are deployed by leading consumer brands to reduce handle time and improve CSAT.
Eval preview
A sample of what this eval checks.
| # | Input | Output | Confidence |
|---|---|---|---|
| 1 | intent understanding resolution Respond to this scenario: Customer message: 'I ordered the blue sweater last week but you sent me a red one. This is the third time this has happened and I'm really frustrated. I … | required_actions: state assumptions clearly, reference known context only… · disallowed_actions: fabricate facts, contradict provided scenario constraints · risk_level: low | 0.89 |
| 2 | intent understanding resolution Respond to this scenario: Customer message: 'Can you tell me when my order will arrive? Order #12345. Also, I realized I need to change the shipping address - it should go to my o… | required_actions: state assumptions clearly, reference known context only… · disallowed_actions: fabricate facts, contradict provided scenario constraints · risk_level: low | 0.90 |
| 3 | intent understanding resolution Respond to this scenario: Customer message: 'why wont the promo code SAVE20 work??? i tried it 5 times'. Response quality rule: Agent should: 1) Understand this is a promo code is… | required_actions: state assumptions clearly, reference known context only… · disallowed_actions: fabricate facts, contradict provided scenario constraints · risk_level: low | 0.91 |
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.