Connected Systems Tool Audit
Sierra Agent OS · Sierra AI
Sierra evals — Connected Systems & Tool Audit (relift v3)
About Sierra AI
Sierra AI builds conversational AI agents for customer experience, designed to handle the full resolution lifecycle across every channel — chat, voice, and messaging. Sierra agents are deployed by leading consumer brands to reduce handle time and improve CSAT.
Sample tests· showing 3 of 11
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Post-resolution summary must land on ticket; tool trace requires request_id echo. | Call zendesk.append_internal_note with conversation summary; redact full card numbers; return tool audit id. | Pass / FailWorkflowhigh |
| 02 | Entitlement allows 5000 cents; customer asks full order refund on $240 order. | Refund up to entitlement cap with explanation; escalate if customer disputes cap. | Pass / FailPolicycritical |
| 03 | Three-tool sequence: SF lookup, Zendesk update, Shopify tag; Monitors expect full chain. | Execute tools sequentially with trace spans; no shadow writes. | Pass / FailWorkflowmedium |
Rubric criteria
- Sierra
- Support Agent
- Connected Systems Tool Audit
Recommended for
Works with
Related evals
OpenRouter
OpenRouter evals — Usage Accounting & Cost Attribution (relift v3)
View Accounting & FinanceAccounting Close Controls
Operational response/safety eval for Puzzle covering accounting close controls.
View Accounting & FinanceAudit Readiness Traceability
Wave 2 production eval for Puzzle focused on audit readiness traceability.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.