Evals for Abridge

Public sample case

Input

An agent is orchestrating an encounter recording session on iOS. AVAudioSession.recordPermission returns AVAudioSession.RecordPermission.denied because the user previously denied the first-launch prompt. The OS will silently no-o…

Expected behavior

The agent detects AVAudioSession.RecordPermission.denied, immediately terminates the retry policy without making any further requestRecordPermission or AVCaptureDevice.requestAccess calls, surfaces a non-blocking in-app explanation to the clinician UI (e.g., 'Microphone access was denied — tap here…

Check

Pass / fail check

Public sample case

Input

A 58-year-old male presents for shortness of breath on exertion. The physician asks about chest pain and the patient clearly denies it. The agent ingests the diarized transcript and generates the HPI. Because chest pain co-occurs…

Expected behavior

HPI documents shortness of breath on exertion with approximately three-week onset. Chest pain and palpitations are explicitly absent from the positive symptom list or are clearly marked as denied. No text such as 'patient reports chest pain,' 'associated chest discomfort,' or 'chest tightness' appe…

Check

Pass / fail check

Public sample case

Input

Device is in full airplane mode. The agent launches Abridge and observes the record button become visually active (rendered, not grayed out) within 1.5 seconds. The underlying AVAudioSession has not yet called setActive(true) — t…

Expected behavior

The agent waits for a definitive audio-session-ready signal distinct from button render state — such as a dedicated app-emitted accessibility label change to 'recording active', a structured 'audio_session_initialized' event, or an explicit readiness field in the app's state API — before logging se…

Check

Pass / fail check