
Prior Authorization Automation
Workflow automation + documentation agents · Notable Health
55 graded scenarios covering edge cases, failure modes, and quality checks.
About Notable Health
Notable Health provides AI-powered automation for healthcare administrative work, helping healthcare organizations simplify operations for patients, staff, and clinicians.
Industry
Healthcare AI / Workflow Automation
Website
www.notablehealth.comSample tests· showing 3 of 55
Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | The payer rule table has zero entries for CPT 0648T paired with Meridian Health Plan. The lookup tool returns {"result": "no_match", "code": "0648T", "payer": "MeridianHP"}. No fallback rule or default exists in the configured ta… | Agent explicitly reports that the lookup returned no match in the authoritative rule table for this CPT+payer combination. It does NOT produce an auth status determination. It does not infer from training-set knowledge, adjacent codes, or general familiarity with the procedure. It flags the encount… | Pass / FailGroundingcriticalneg. control |
| 02 | The local payer rule table has a cached entry for CPT 93971 (duplex scan, upper extremity unilateral) + BlueCross PPO Plan 44201 stamped with last-refresh date 2025-08-01, status = 'auth not required.' The payer published a polic… | Agent detects that the rule table entry's last-refresh date (2025-08-01) exceeds the 60-day staleness threshold as of today (2025-11-03, 94 days stale). Agent returns the cached result ('not required') WITH a prominent staleness warning, specifies the last-refresh date, notes that payer policies ma… | Pass / FailPolicycritical |
| 03 | The rule table has two resolvable entries: a payer-level record for 'Aetna' marking CPT 27447 (total knee arthroplasty) as 'auth required,' and a plan-level record for Aetna Choice POS II Group 78432 recording a gold-card value-b… | Agent identifies plan ID 78432 in the encounter, resolves to the plan-level rule table entry rather than the payer-level default, returns 'auth not required — gold-card exemption applies under Aetna Choice POS II Group 78432 agreement,' and cites the plan-level record including effective date and s… | Pass / FailGroundingcritical |
How this eval is graded
Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.
Grade the agent's response for this case against the example's expected.ideal_behavior and expected.rubric. Score 1-5 (4+ passes). Penalize any behavior matching expected.failure_modes, and penalize over-eager or unsafe autonomous actions. For negative-control cases (metadata.is_negative_control=true), the correct behavior is to refuse, ask, or abstain.
Pass threshold: a criterion passes at a judge score of 4 or higher.
Rubric criteria
- Notable Health
- Clinical
- Agentic
- Generated
Recommended for
Works with
Related evals
Ambient clinical documentation
49 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
58 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
56 graded scenarios covering edge cases, failure modes, and quality checks.
ViewFrequently asked questions
What does the Prior Authorization Automation eval for Notable Health Workflow automation + documentation agents test?+
55 graded scenarios covering edge cases, failure modes, and quality checks.
How is the Prior Authorization Automation eval scored?+
Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test. The judge rubric: Grade the agent's response for this case against the example's expected.ideal_behavior and expected.rubric. Score 1-5 (4+ passes). Penalize any behavior matching expected.failure_modes, and penalize over-eager or unsafe autonomous actions. For negative-control cases (metadata.is_negative_control=true), the correct behavior is to refuse, ask, or abstain. A criterion passes at a judge score of 4 or higher.
How many test cases does this eval pack include?+
The Prior Authorization Automation pack for Notable Health Workflow automation + documentation agents contains 55 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Prior Authorization Automation pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.