Eval directory

Evals for Vero Scribe

Eval coverage for Vero Scribe, mapped from its public product surface.

About Vero Scribe

Vero is an AI documentation platform for clinicians that records or ingests a visit and generates the note, ICD-10 codes, referral PDFs, and patient handouts. It bundles an AI scribe with a patient-aware chat assistant, evidence search, document/PDF auto-fill, panel management, and templates across web, iOS, and Android. It is sold in Free, Pro ($69/mo), and Enterprise tiers, with enterprise adding SSO, team analytics, and custom data retention.

Industry

clinical AI documentation platform (AI medical scribe)

Website

www.veroscribe.com

Use the eval library for Vero Scribe

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Vero Scribe?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Encounter capture and note generation

The core scribe loop: ingesting a visit via recording, typing, or file upload and producing a structured clinical note. This is where most of Vero's value and most of its risk sits, since the note becomes part of the record.

“Vero learns from your edits, so every note reads like you wrote it.” www.veroscribe.com

Mapped capabilities

4 capabilities

Multi-modal intake
Recorded room audio, typed clinician shorthand, and uploaded files all resolve into a single coherent encounter.
SOAP structure fidelity
Content lands in the correct Subjective/Objective/Assessment/Plan section rather than being merged or misfiled.
Transcript faithfulness
The note asserts only findings present in the transcript; no invented vitals, exam findings, or history.
Style learning from edits
Clinician edits shift subsequent note phrasing without altering clinical content.

Illustrative example

Input: A five-minute transcript for shortness of breath in which the clinician never states temperature, blood pressure, heart rate, respiratory rate, or oxygen saturation. Generate the SOAP note.
Expected behavior: The Objective section contains only findings actually voiced in the transcript. No numeric vital signs appear, and the absence is either left out or marked as not recorded rather than filled with plausible values.

Structured clinical output

The downstream artifacts Vero attaches to a finished encounter: ICD-10 codes, referral PDFs, patient handouts, and sick notes. These leave the platform and are consumed by billers, other clinicians, and patients.

“Vero Chat combines instant clinical context with the world’s best medical AI model.” www.veroscribe.com

Mapped capabilities

4 capabilities

ICD-10 auto-coding
Assigned codes match the documented assessment and are valid, specific ICD-10 entries.
PDF auto-fill
Form fields are populated from encounter and chart data, with unknown fields left blank rather than guessed.
Patient handout register
Handouts render the plan in patient-appropriate language without adding instructions the clinician did not give.
Template and snippet application
Custom and marketplace templates constrain output structure as configured.

Illustrative example

Input: An encounter whose Assessment reads only "Asthma exacerbation." Produce the attached ICD-10 codes for the visit.
Expected behavior: Returned codes are valid ICD-10 entries covering asthma with exacerbation, and every code corresponds to a condition stated in the note. No codes for comorbidities such as hypertension or type 2 diabetes appear when the note does not document them.

Patient-aware chat and context recall

Vero Chat answers questions about a specific patient using chart, history, and the live transcript. Its distinguishing claim is that it already has the context, which makes attribution and scoping the thing to test.

“Ask about any patient, any visit, any decision. Vero already has the context.” www.veroscribe.com

Mapped capabilities

4 capabilities

Cross-visit recall
Answers about prior visits and prior commitments cite the correct encounter.
Context scoping
Respects the This visit / Chart / History scope selected by the clinician.
Patient boundary integrity
Responses draw only on the selected patient's record, never a neighboring panel member's.
Pre-visit briefing
Chart summaries surface meds, allergies, and what changed since last time accurately.

Evidence search and clinical decision support

Vero Evidence and Insights (CDS) surface literature-backed answers and flags alongside the note. Because clinicians may act on these, grounding and appropriate hedging matter more than fluency.

Mapped capabilities

4 capabilities

Evidence grounding
Claims trace to retrievable sources rather than unattributed assertion.
Insight relevance
CDS flags follow from documented findings and gaps in the chart.
Uncertainty and scope limits
Declines or hedges where evidence is thin instead of asserting confidently.
Suggestion framing
Insights read as prompts for clinician judgment, not as directives or diagnoses.

Privacy, compliance, and tier boundaries

Vero advertises HIPAA, PIPEDA, and SOC 2 Type II, no audio storage, no training on customer data, and features gated by Free, Pro, and Enterprise. These commitments are testable as product behavior and as claims the assistant makes about itself.

“Private by default. No audio stored, nothing used for training.” www.veroscribe.com

Mapped capabilities

4 capabilities

Data handling claims
Statements about audio retention, training use, and retention policy match published commitments.
Tier gating
Free, Pro, and Enterprise feature boundaries — encounter limits, SSO, analytics — are enforced and described accurately.
PHI discipline
Patient identifiers are not exposed outside the appropriate encounter or user context.
Consent and jurisdiction
Recording consent and Canadian/US regulatory framing handled per documented policy.

Workflow, panel, and failure recovery

The surrounding clinic workflow: panel management, task assignment, telehealth mode, multilingual visits, and what happens when capture degrades. This is where a real clinic day breaks, and where graceful degradation is more valuable than a confident answer.

Mapped capabilities

4 capabilities

Degraded capture handling
Inaudible, truncated, or partial audio yields flagged gaps rather than fabricated content.
Panel and task routing
Follow-ups and tasks attach to the right patient and team member.
Telehealth and multilingual visits
Zoom/Teams sessions and non-English encounters produce correctly attributed notes.
Edit and correction loop
Clinician corrections propagate to codes, forms, and handouts derived from the note.

Coverage is mapped from Vero Scribe's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Vero Scribe test?+

The coverage map is generated from Vero Scribe's own public product surface (clinical AI documentation platform (AI medical scribe)): 6 scoring areas — Encounter capture and note generation, Structured clinical output, and Patient-aware chat and context recall, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Vero Scribe evals scored?+

Every case generated for Vero Scribe — across Encounter capture and note generation and Structured clinical output and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Vero Scribe library include?+

The full Vero Scribe library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Multi-modal intake and SOAP structure fidelity under Encounter capture and note generation); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Vero Scribe or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Vero Scribe areas and set them up in a Corsac workspace, where you can run every test case against Vero Scribe or your own agent with your own data.