Evals for ASAPP

Intent understanding across multi-turn conversations

Correctly identifying and holding the customer's actual issue as it is clarified or amended over several turns.

Taking action to resolve, not just answering

Executing the request against connected systems where the interaction type supports automation, rather than restating policy or deflecting.

Channel-appropriate behavior on voice vs. chat

Adapting response form and pacing to the channel while resolving the same underlying request.

Personalized handling using known customer context

Using the customer context available to the agent so the interaction reflects what is already known about them.

Recognizing when to involve a human agent

Escalating on ambiguity, sensitivity, or out-of-scope requests instead of proceeding unsupported.

Context transfer on escalation

Passing the issue summary, prior turns, and actions already taken so the human agent does not restart the conversation.

Resuming after human involvement

Picking the interaction back up without contradicting or repeating what the human agent handled.

Agent-facing assistance during live interactions

Supporting the human agent's work on the interaction rather than competing with it.

Following defined steps in order

Completing required steps of a configured flow without skipping or reordering them.

Staying in flow under conversational drift

Returning to the procedure when the customer digresses mid-flow.

Handling flow prerequisites and blocked steps

Behaving predictably when a required input or precondition is missing.

Boundary between free-form reasoning and deterministic steps

Using natural conversation where appropriate while deferring to the deterministic path where one is defined.

Retrieving customer and account data from connected systems

Grounding responses in system-of-record data rather than assumption.

Writing back actions and outcomes

Recording the resolution in connected systems consistently with what the customer was told.

Behavior when an integration is unavailable

Degrading transparently when a connected system cannot be reached, rather than fabricating a result.

Operating across channels and systems in one interaction

Keeping a single coherent interaction when more than one connected system is involved.

Respecting configured scope and automation boundaries

Declining requests outside the use cases the deployment is authorized to automate.

Sensitive data handling in conversation

Handling customer identifiers and account details according to configured controls.

Refusing unsafe or out-of-policy actions

Holding the line when a customer pressures the agent toward an action policy does not permit.

Identity and authorization before consequential actions

Confirming the requester is entitled to the action before executing it.

Traceability of agent reasoning and actions

Producing a reviewable record of why the agent did what it did in an interaction.

Pre-deployment testing of agent behavior

Exercising configured behavior against representative interactions before it reaches customers.

Monitoring live agent performance

Surfacing how the deployed agent is performing on resolution outcomes over time.

Use case selection and automation potential

Supporting the data-driven identification of interaction types with high automation potential.