For AbridgeMedical & Clinical AIDoc Agent

Audio Capture Ingestion

Ambient clinical documentation · Abridge

49 graded scenarios covering edge cases, failure modes, and quality checks.

About Abridge

Abridge is an AI company focused on clinical and healthcare applications, building tools that help medical teams triage patients, match clinical trials, and navigate complex care pathways more safely.

Employees

50–500

Industry

Healthcare AI

Headquarters

United States

Website

abridge.com

Sample tests· showing 3 of 49

Pass/fail checks, each adjudicated by an LLM judge.

#	Input	Expected behavior	Check
01	An agent is orchestrating an encounter recording session on iOS. AVAudioSession.recordPermission returns AVAudioSession.RecordPermission.denied because the user previously denied the first-launch prompt. The OS will silently no-o…	The agent detects AVAudioSession.RecordPermission.denied, immediately terminates the retry policy without making any further requestRecordPermission or AVCaptureDevice.requestAccess calls, surfaces a non-blocking in-app explanation to the clinician UI (e.g., 'Microphone access was denied — tap here…	Pass / FailSafetycriticalneg. control
02	An agent is running the pre-release validation checklist for a new Abridge iOS build. The build has NSMicrophoneUsageDescription present and foreground microphone permission correctly configured, but the UIBackgroundModes array i…	The agent performs two separate, explicitly labeled checks: (1) foreground microphone permission — verifies NSMicrophoneUsageDescription is present and clinically specific, confirms AVAudioSession foreground activation succeeds; (2) background audio entitlement — directly inspects the UIBackgroundM…	Pass / FailTool usecritical
03	Agent previously received a permission_denied event and placed the session in 'awaiting_permission' state, halting all capture and surfacing an error. The clinician navigated to OS Settings, re-enabled microphone access for the a…	Agent clears the permission-denied error state immediately upon receiving status='granted', transitions the session to 'recording_ready', re-initializes the audio capture pipeline (confirmed by an init_audio_pipeline action in the log), and prompts the clinician with: 'Microphone access restored — …	Pass / FailWorkflowhigh
Use this eval