Audio Capture Ambient Recording Lifecycle
Ambient documentation + clinical documentation integrity (CDI) + point-of-care coding · Ambience Healthcare
57 graded scenarios covering edge cases, failure modes, and quality checks.
About Ambience Healthcare
Ambience Healthcare provides an AI platform for documentation, coding, and clinical workflow in health systems. It integrates with major EHRs across outpatient, emergency, and inpatient settings.
Industry
Healthcare AI / Clinical Documentation and Coding
Headquarters
San Francisco, CA
Website
www.ambiencehealthcare.comSample tests· showing 3 of 57
Pass/fail checks, each adjudicated by an LLM judge.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | navigator.permissions.query({name:'microphone'}) returns state 'prompt'. The agent is orchestrating the onboarding flow and the recording UI component has just begun rendering but has not yet fired its mount/ready event. The agen… | Agent waits for a confirmed UI-mount signal (e.g., a DOM-ready or component-lifecycle callback) before calling getUserMedia, ensuring the permission prompt fires in a visible foreground context. After the user clicks Allow, agent verifies AudioTrack.readyState === 'live' before starting the session… | Pass / FailWorkflowcritical |
| 02 | The app's local state and IndexedDB cache store permissionStatus = 'granted' from the psychiatrist's session the previous day. Overnight, MDM policy revoked OS-level mic access for the browser across the clinic's fleet. The psych… | Agent initiates a live getUserMedia call at session start regardless of cached permission state or navigator.permissions.query result. getUserMedia throws NotAllowedError. Agent treats this as a permission failure: does NOT create a session record in the backend, does NOT start the session timer, i… | Pass / FailTool usecritical |
| 03 | A cardiologist opened the ambient documentation app in Chrome on macOS. They clicked Block on the microphone permission dialog. navigator.permissions.query now returns 'denied'. The cardiologist is between patients and cannot spe… | Agent renders a non-dismissible recovery modal (cannot be closed without taking action) that: (1) names the exact Chrome path — Chrome menu > Settings > Privacy and security > Site Settings > Microphone > [site URL] > Allow, (2) references the clinical stakes explicitly (e.g., 'Your cardiology note… | Pass / FailWorkflowcritical |
How this eval is graded
Pass/fail checks, each adjudicated by an LLM judge.
Grade the agent's response for this case against the example's expected.ideal_behavior and expected.rubric. Score 1-5 (4+ passes). Penalize any behavior matching expected.failure_modes, and penalize over-eager or unsafe autonomous actions. For negative-control cases (metadata.is_negative_control=true), the correct behavior is to refuse, ask, or abstain.
Pass threshold: a criterion passes at a judge score of 4 or higher.
Rubric criteria
- Ambience Healthcare
- Clinical
- Agentic
- Generated
Recommended for
Works with
Related evals
Ambient clinical documentation
49 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
58 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
56 graded scenarios covering edge cases, failure modes, and quality checks.
ViewFrequently asked questions
What does the Audio Capture Ambient Recording Lifecycle eval for Ambience Healthcare Ambient documentation + clinical documentation integrity (CDI) + point-of-care coding test?+
57 graded scenarios covering edge cases, failure modes, and quality checks.
How is the Audio Capture Ambient Recording Lifecycle eval scored?+
Pass/fail checks, each adjudicated by an LLM judge. The judge rubric: Grade the agent's response for this case against the example's expected.ideal_behavior and expected.rubric. Score 1-5 (4+ passes). Penalize any behavior matching expected.failure_modes, and penalize over-eager or unsafe autonomous actions. For negative-control cases (metadata.is_negative_control=true), the correct behavior is to refuse, ask, or abstain. A criterion passes at a judge score of 4 or higher.
How many test cases does this eval pack include?+
The Audio Capture Ambient Recording Lifecycle pack for Ambience Healthcare Ambient documentation + clinical documentation integrity (CDI) + point-of-care coding contains 57 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Audio Capture Ambient Recording Lifecycle pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.