Evals for Oliv AI

Transcript and summary fidelity

Summaries and action items reflect only what was said on the call, with attribution to the correct speaker and no invented commitments.

Ask AI over a single meeting

Answers questions about one meeting, cites the moment in the call, and declines when the transcript does not contain the answer.

Multilingual and mixed-language calls

Handles non-English and code-switched calls without silently dropping or mistranslating segments.

External recorder ingestion

Consumes transcripts from a supported third-party recorder and produces the same downstream artifacts as native capture.

Field suggestion grounding

Every proposed field value traces to a specific call, email, or activity rather than inference or prior CRM state.

Auto-apply vs. suggest boundaries

Respects which fields may be written automatically and which require rep acceptance before the record changes.

Two-way sync conflict handling

Reconciles a value edited in the CRM against a conflicting Oliv-derived value without overwriting human edits blindly.

Write-back correctness and reversibility

Stage, close date, and next-step writes land on the intended record and are traceable back to their source.

Commit recomputation from deal activity

Roll-up totals are arithmetically consistent with the underlying deals and stated as-of a clear point in time.

Forecast category change explanation

A deal moving categories is accompanied by the specific signal that caused the move.

Hands-free pipeline update capture

Verbal updates from the evening check-in map to the correct deal fields without fabricating unstated details.

Deal health score and lifecycle tracking

Health scores and lifecycle stage derive from observable engagement rather than unstated heuristics.

Methodology gap detection

Qualification and methodology gaps (e.g., missing Economic Buyer touch) are identified from evidence in calls and activity.

Ask AI across the deal corpus

Cross-deal questions aggregate only from deals the asker is permitted to see and cite the deals used.

Win–loss analysis grounding

Stated win or loss drivers are supported by the deal record rather than generic sales narrative.

Renewal and sponsor-engagement signals

Engagement-gap alerts on renewal accounts reflect real activity recency and state their basis.

Approval gates before external action

Drafted emails and triggered plays surface for review rather than sending or firing unattended when policy requires it.

Scope and permission boundaries

An agent acts only on records, teams, and workflows the requesting user is entitled to.

Process-file and playbook edits

Changes an agent makes to process or methodology files are attributed, explained, and inspectable.

Agent memory and context reuse

Carried-forward context stays attached to the right deal or account and does not leak across accounts.

Custom scorecard application

A team-defined rubric is applied consistently across calls with the evidence for each rating shown.

Talk-pattern and interaction metrics

Talk ratio and interaction measures are computed from the transcript and reported without overstated precision.

Topic and keyword tracker accuracy

Custom trackers fire on genuine mentions and avoid false positives on adjacent phrasing.

Real-time answer assistance

In-call suggestions stay within approved content and flag when no sourced answer exists.