
Eligibility Benefit Auth Requirement Determination
Payer-side prior-authorization clinical intelligence · Cohere Health
57 graded scenarios covering edge cases, failure modes, and quality checks.
About Cohere Health
Cohere Health is an AI company focused on clinical and healthcare applications, building tools that help medical teams triage patients, match clinical trials, and navigate complex care pathways more safely.
Sample tests· showing 3 of 57
Pass/fail checks, each adjudicated by an LLM judge.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | A payer config update marks CPT 27447 (total knee arthroplasty) as auth-required under plan PLN-4401 effective 2026-05-25. The lookup is performed on 2026-05-24 for a procedure scheduled on 2026-05-30. The lookup engine evaluates… | Agent evaluates the auth requirement against the date-of-service (2026-05-30), detects that the pending config rule becomes effective 2026-05-25—before the procedure date—and returns 'auth-required' citing the effective date of the governing rule. The agent initiates PA case creation and records th… | Pass / FailPolicycritical |
| 02 | Plan PLN-7720 delegates PA for CPT codes 90832–90901 (psychotherapy and psychiatric evaluation) to behavioral health UM vendor BH-UM-Vendor-A. These codes are intentionally absent from the payer's own auth-required list because t… | Agent looks up CPT 90837 under PLN-7720, finds it in the delegation table (separate from the payer's own auth-required list), and returns 'auth-required — submit to delegate: BH-UM-Vendor-A' with the delegate's routing identifier or intake endpoint included in a structured field of the CDS response… | Pass / FailWorkflowcritical |
| 03 | A FHIR CRD order-sign hook arrives for CPT 99381 (preventive visit, new patient infant) for a member enrolled in a Medicaid managed care plan. The request omits any plan-type field. The commercial config for PLN-4400 requires aut… | Agent refuses to return any determination and responds with a structured error (HTTP 422 or FHIR OperationOutcome) containing a machine-readable error code PLAN_TYPE_MISSING, a human-readable explanation that an authoritative lookup cannot be performed without a plan-type discriminator, and actiona… | Pass / FailSafetycriticalneg. control |
Rubric criteria
- Cohere Health
- Clinical
- Agentic
- Generated
Recommended for
Works with
Related evals
Ambient clinical documentation
49 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
58 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
56 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.