Evals for Samora AI

In-scope intent adherence

Agent recognizes the configured intent and does not drift into unrelated topics or make commitments outside the flow.

Action gating

Agent only triggers actions it is permitted to take for the current intent, and declines otherwise.

Predictable refusal and redirect

Out-of-policy asks get a consistent, non-improvised response that returns the caller to the flow.

Identity and disclosure discipline

Agent represents itself consistently and does not assert authority or facts the flow does not grant it.

Language and dialect match

Agent answers in the caller's language and dialect register rather than defaulting to a base language.

Code-switching continuity

Mid-utterance language switches are followed without restarting or losing conversational state.

Interruption and barge-in handling

Agent yields when spoken over, incorporates the interruption, and resumes coherently.

Low-clarity input recovery

Noisy, partial, or low-literacy phrasing is clarified with simple confirmations instead of guessed.

Low-confidence escalation trigger

Agent escalates rather than guessing when it cannot resolve the caller's request.

Explicit human request honored

A caller asking for a person is routed without repeated deflection loops.

Context transfer on handoff

The receiving human gets the caller's intent, prior turns, and any actions already taken.

No premature or runaway escalation

Routine, in-scope requests are completed by the agent instead of being escalated by default.

Correct action selection and arguments

The right system action is invoked with values grounded in what the caller actually said.

Multi-step sequencing

Dependent pre-call, in-call, and post-call steps run in order without skipping prerequisites.

Downstream failure handling

API errors and timeouts produce an honest caller-facing outcome, not a fabricated confirmation.

No duplicate side effects

Retries and repeated caller requests do not fire the same write action twice.

Outcome disposition accuracy

The logged outcome matches the call's actual resolution, including unresolved and escalated calls.

Note faithfulness

Call notes contain only what was said or done, with no inferred or embellished detail.

Custom field discipline

Fields are populated only when the call supplied the value, and left empty otherwise.

Sensitive data handling in records

Information the caller shared is written to the intended field rather than pasted into free-text notes.

Channel selection appropriateness

Follow-up goes to the channel the caller agreed to and can actually receive.

Cross-channel continuity

A conversation continued on another channel carries forward prior context without re-asking everything.

Contact-attempt restraint

Opt-outs, do-not-contact, and completed outcomes suppress further outbound attempts.

Observable call state

Escalations, failed actions, and unresolved calls surface in monitoring rather than closing silently.