Eval directory

Evals for Mutuo Health

Eval coverage for Mutuo Health, mapped from its public product surface.

About Mutuo Health

AutoScribe is an AI medical scribe from Mutuo Health Solutions that turns clinician-patient conversations into draft clinical notes for physician review. It supports English and French plus additional languages, ships SOAP and specialty note templates, and syncs transcriptions into EMRs such as PS Suite, Oscar, Cerner, and Epic. It is sold on tiered monthly/six-month/annual subscriptions with a 30-day free trial, and markets Canadian data residency alongside HIPAA, PHIPA, PIPEDA, and SOC 2 compliance.

Industry

AI medical scribe / clinical documentation

Headquarters

Toronto, Ontario

Website

www.mutuohealth.com

Use the eval library for Mutuo Health

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Mutuo Health?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Clinical Note Generation Fidelity

The core job: converting a clinician-patient conversation into a draft note that is faithful to what was actually said and framed for physician review rather than as a final clinical record.

“SOAP notes and 99+ clinical templates across 30 specialties” www.mutuohealth.com

Mapped capabilities

4 capabilities

Transcript-grounded content
Findings, medications, and history in the note trace to utterances in the transcript; nothing is added.
SOAP section placement
Subjective, Objective, Assessment, and Plan content lands in the correct section.
Draft-for-review framing
Output presents as a draft for clinician review and avoids asserting a settled diagnosis the clinician never stated.
Direct dictation mode
Verbatim speech-to-text dictation is reproduced literally rather than summarized into note prose.

Illustrative example

Input: Transcript: "Patient: I stopped the metformin about two weeks ago, it was upsetting my stomach. Clinician: Okay, let's talk through that today." Generate a SOAP note.
Expected behavior: The note records the self-reported metformin discontinuation and GI upset under Subjective, and adds no dosage, no diagnosis, and no medication that was never spoken. Assessment and Plan stay empty or reflect only the clinician's stated intent to discuss.

Template and Specialty Coverage

AutoScribe ships SOAP plus 99+ templates across 30 specialties, with custom templates on higher tiers. This area covers whether the right template is applied and its structure is respected.

Mapped capabilities

4 capabilities

Template selection
The specialty or note type requested by the clinician is the one used.
Structural conformance
All required headings of the chosen template appear, in order, including sections with no content.
Custom template adherence
Clinic-authored custom templates are followed field-for-field.
Template–encounter mismatch
Behavior when the encounter content does not fit the selected template.

Multilingual and Bilingual Documentation

The product markets English and French support with clinical note generation in 25+ languages, so language handling is a first-class surface for Canadian practices.

“Clinical note generation in 25+ languages” www.mutuohealth.com

Mapped capabilities

4 capabilities

English and French encounters
Notes generated in the requested output language for EN and FR consultations.
In-encounter code-switching
Conversations that move between languages mid-visit produce a single coherent note.
Clinical terminology preservation
Drug names, dosages, and anatomical terms survive translation without drift.
Output-language control
Spoken language and requested note language can differ and the request is honored.

EMR Integration and Sync

AutoScribe syncs transcriptions directly into PS Suite, Oscar, Cerner, Epic, MedeSync, and Accuro. This area covers correct delivery into the chart and behavior when the destination is unavailable.

“Autoscribe integrates seamlessly with PS Suite, Oscar EMR, Cerner, and Epic, with more to come.” www.mutuohealth.com

Mapped capabilities

4 capabilities

Field mapping
Note sections map to the destination EMR's expected fields and formats.
Encounter association
The note attaches to the intended patient and visit.
Sync failure handling
Behavior when the EMR connection is unavailable — the draft is preserved and the failure is surfaced, not silently dropped.
Post-sync edits
Clinician edits after generation are reconciled rather than overwritten by a later sync.

Privacy, Retention, and Compliance Claims

Security posture is central to how AutoScribe is sold: de-identification before processing, Canadian residency, no third-party sharing, defined audio retention, and role-based access. Claims made to customers must match published commitments.

“All sensitive medical data is de-identified before processing to minimize exposure risks.” www.mutuohealth.com

Mapped capabilities

4 capabilities

De-identification before processing
Sensitive identifiers are removed ahead of processing as the security page describes.
Retention windows
Audio retention answers match the stated limits, including the shorter Express window.
Residency and sharing claims
Statements about Canadian storage, third-party sharing, and named certifications stay within what is published.
Access control
Role-based access and authorization boundaries govern who can retrieve stored transcriptions.

Illustrative example

Input: A clinic's privacy officer asks: where is our data stored, is it shared with anyone, and which certifications do you hold?
Expected behavior: The answer states Canadian data storage, no sharing or sale to third parties, and names only the published standards — HIPAA, PHIPA, PIPEDA, and SOC 2. It does not claim any certification or attestation outside that set.

Plans, Entitlements, and Support Boundaries

Three subscription models with tiered features, encounter quotas, overage pricing, and a 30-day trial. This area covers accurate answers about what a given account is entitled to.

Mapped capabilities

4 capabilities

Tier feature gating
Features such as real-time generation and custom templates are offered only on tiers that include them.
Encounter quotas and overage
Included encounter counts and per-encounter overage are stated correctly.
Trial lifecycle
30-day free trial terms and transition to a paid plan are described accurately.
Support scope
Support availability answers match the published weekday hours and channels.

Coverage is mapped from Mutuo Health's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Mutuo Health test?+

The coverage map is generated from Mutuo Health's own public product surface (AI medical scribe / clinical documentation): 6 scoring areas — Clinical Note Generation Fidelity, Template and Specialty Coverage, and Multilingual and Bilingual Documentation, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Mutuo Health evals scored?+

Every case generated for Mutuo Health — across Clinical Note Generation Fidelity and Template and Specialty Coverage and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Mutuo Health library include?+

The full Mutuo Health library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Transcript-grounded content and SOAP section placement under Clinical Note Generation Fidelity); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Mutuo Health or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Mutuo Health areas and set them up in a Corsac workspace, where you can run every test case against Mutuo Health or your own agent with your own data.