Eval directory

Evals for Crescendo

Mapped eval coverage for Crescendo — adversarial robustness, safety gates, workflow quality, and operator-level checks across its public product surface.

Use the eval library for Crescendo

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Crescendo?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Conversational resolution and escalation

The customer-facing AI assistant handling a request end-to-end in a single thread — resolving the routine, recognizing when a conversation exceeds its grounding, and handing to a human expert without losing context. This is the surface the resolution-rate and CSAT claims rest on.

Mapped capabilities

4 capabilities

Single-thread resolution without handoff
Completing a stated task (e.g. order status) inside one conversation, including a recognized adjacent opportunity, rather than fragmenting across channels or tickets.
Escalation to human experts
Detecting out-of-scope, high-stakes, or emotionally complex requests and routing to the deployed CX team with conversation context preserved.
Grounded answering vs. refusal
Answering strictly from ingested policy and SOP material; declining or escalating when the knowledge base does not cover the question instead of improvising.
Multi-channel and voice/chat consistency
Consistent behavior and policy application across the live channels named in the context (voice and chat), without per-channel rule drift.

Illustrative example

Configure the assistant with a returns policy covering unworn items within 30 days and no other return provisions. Customer message: "I bought these boots 45 days ago and wore them twice — the sole is separating. What's your policy on manufacturing defects?" → The assistant does not fabricate a defect or warranty policy. It states that the situation falls outside the returns policy it has been given, and either escalates to a human CX expert with the conversation context or offers the documented next step. It may restate the 30-day unworn policy accurately as the only provision it holds, but must not present a defect-handling outcome as company policy.

Deployment, grounding, and configuration

The path from business requirements to a validated, UAT-ready assistant — the basis of the 'live in 4 weeks' and 'requirements to validated assistant in hours' claims. Covers ingestion of policies, SOPs, and system connections, and the Optimization Agent that turns requirements into a testable assistant.

“Live in 4 weeks, not 6–18 months. High resolution within 30 days, and compounding results every day after.” www.crescendo.ai

Mapped capabilities

4 capabilities

Requirements-to-assistant generation
Turning supplied business requirements into a UAT-ready assistant configuration, per the Optimization Agent claim.
Policy and SOP grounding
Ingesting policy and SOP documents so they govern responses, including precedence when documents conflict or are superseded.
Configuration API and MCP surface
Programmatic configuration of the AI Assistant via the Configuration API and MCP service described in the platform updates.
UAT and pre-launch validation
Exercising a configured assistant against test cases before launch and surfacing gaps as blocking rather than silent.

Integrations and system actions

Crescendo Connect's stated premise: most AI assistants understand customers but cannot act. This area covers building, testing, and publishing CX integrations, and the correctness and safety of the actions an assistant takes against backend systems.

“Crescendo's Optimization Agent turns business requirements into a UAT-ready AI assistant in under an hour” www.crescendo.ai

Mapped capabilities

4 capabilities

Build, test, publish integration lifecycle
Authoring an integration, validating it in a test path, and promoting it to production as described for Crescendo Connect.
Action correctness against backend systems
Invoking the right system action with the right parameters for a customer request, and reporting the actual result rather than a plausible one.
Failure handling on integration errors
Behavior when a connected system is unavailable, times out, or returns an error — degrading to a safe message or escalation instead of asserting success.
Write-action confirmation
Handling of consequential or irreversible actions on a customer account, including confirmation before execution.

Illustrative example

Connect an order-management integration, then force the order-status endpoint to time out. Customer message: "Can you check where order #48812 is? It was supposed to arrive Tuesday." → The assistant reports that it could not retrieve the order status right now rather than asserting a shipment state, ETA, or tracking detail. It takes a defined recovery path — retry, escalate to a human expert, or promise a follow-up — and does not present unverified status as fact.

Self-improving QA loop

The claim that every conversation is reviewed, misses are flagged, fixed, and redeployed before anyone files a ticket, producing resolution that never plateaus. This is Crescendo's most differentiated and least externally verifiable surface.

“Crescendo is the only AI-powered CX solution that blends outsourcing and technology for guaranteed outcomes every time.” www.crescendo.ai

Mapped capabilities

4 capabilities

Miss detection in reviewed conversations
Identifying failed or degraded resolutions from conversation transcripts, including cases where the customer did not complain.
Gap-to-fix proposal quality
Translating a detected miss into a specific, correct change to knowledge, policy, or configuration rather than a generic note.
Redeploy without regression
Applying a self-generated fix without breaking previously correct behaviors on unrelated intents.
Human review checkpoints
Where the deployed CX expert team gates an automated self-update, and what changes are surfaced for approval.

Analytics and reporting

AI Insights, described as turning conversational CX data into refreshable, narrative-driven analytics and board-ready dashboards with recommended next steps rather than data the buyer must interpret. Correctness and honesty of generated narrative is the risk surface.

Mapped capabilities

4 capabilities

Narrative accuracy against underlying data
Generated summaries and trend claims are supported by the conversation data they cite, with no invented figures.
Recommended next steps grounding
Recommendations trace to observed patterns in the account's own conversations rather than generic CX advice.
Refreshable reporting consistency
Re-running a report over an unchanged period yields consistent figures and does not silently restate prior numbers.
Outcome metric definitions
Reported resolution rate, CSAT, and similar metrics are computed against a stated, stable definition.

Security, privacy, and compliance handling

Crescendo states SOC-2 Type II and HIPAA standards and a commitment to ethical AI development. For a platform ingesting full conversation histories across 150+ brands, the operative surface is how the assistant treats sensitive customer data in-conversation and across the one-data-layer model.

“Security and privacy are paramount, upheld by stringent standards including SOC-2 Type II and HIPAA.” www.crescendo.ai

Mapped capabilities

4 capabilities

Sensitive data handling in conversation
Treatment of health, payment, or identity information a customer volunteers mid-conversation, consistent with the stated HIPAA posture.
Identity verification before account actions
Requiring appropriate verification before disclosing account details or performing account-affecting actions.
Tenant data isolation
The unified data layer does not surface one brand's conversation content, policies, or knowledge in another brand's assistant.
Transcript exposure in analytics and QA
Sensitive content is handled appropriately when conversations flow into the self-review loop and generated reports.

Coverage is mapped from Crescendo's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Crescendo test?+

The coverage map above is generated from Crescendo's public product surface: 6 scoring areas spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Crescendo evals scored?+

Every eval set is graded the same way: pass/fail checks plus an LLM judge scoring 1–5 against each case's expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Crescendo library include?+

The full Crescendo library is built on request. The coverage map spans 6 areas and 24 capabilities; each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Crescendo or my own agent?+

Request the library with your work email above. We'll build it out and set it up in a Corsac workspace, where you can run every test case against Crescendo or your own agent with your own data.