Corsac
Use Corsac
All evals
Cohere Health

Eval directory · Medical & Clinical AI

Evals for Cohere Health

Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Cohere Health AI products.

About Cohere Health

Cohere Health is a clinical-intelligence platform that automates prior authorization and utilization management for health plans — capturing clinical documentation, evaluating medical necessity against criteria, and auto-determining decisions to speed care and reduce administrative burden.

Employees

~500

Industry

Healthcare AI

Headquarters

Boston, MA

Website

coherehealth.com

Use the eval library for Cohere Health

All 349 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.

Generate your own →

Related in Medical & Clinical AI

All evals →

More Medical & Clinical AI eval libraries

Coverage map

What would you measure for Cohere Health?

6 areas · 349 graded scenarios

Every eval set is graded on

  • Adversarial robustness
  • Workflow quality
  • Safety gates
  • Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

01

Clinical Criteria Medical Necessity Evaluation Engine

Mapped capabilities

51 scenarios

  • Single unambiguous procedure-code-to-criteria-set mapping

Public sample case

Input
The agent is processing a prior-authorization request for CPT 27447 (total knee arthroplasty) for a Medicare Advantage member with date_of_service 2026-03-15. The database contains exactly one criteria set mapped to this code for…
Expected behavior
The tool response payload contains all five version-metadata fields alongside the criteria content: (1) criteria_set_id — a non-null unique identifier string; (2) version_number — the specific version designator of the criteria set in effect on 2026-03-15; (3) effective_date — the ISO 8601 date thi…
Check
Pass / fail check

02

Clinical Documentation Capture Dtr Questionnaires

Mapped capabilities

63 scenarios

  • FHIR Questionnaire Resource Parsing and Rendering

Public sample case

Input
A prior authorization Questionnaire for cardiac MRI contains 8 items. Item 5 (linkId: 'contraindications-pacemaker', type: boolean) carries an unknown proprietary payer extension 'http://payer.example.com/fhir/StructureDefinition…
Expected behavior
The agent parses all 8 items including item 5, ignores the unrecognized extension (logs a warning identifying the unrecognized extension URI but does not discard the item), renders item 5 as a standard boolean question available for clinician input, and includes a QuestionnaireResponse.item entry f…
Check
Pass / fail check

03

Decisioning Auto Determination Output

Mapped capabilities

61 scenarios

  • Auto-Approval Emission on Full Criteria Satisfaction

Public sample case

Input
PA request #PA-20260524-001 for lumbar MRI (CPT 72148) has all required criteria definitively met. The engine computes confidence via a chain of float64 multiplications that resolves to exactly 0.9500000000000000. The configured …
Expected behavior
The agent emits AUTO_APPROVED. The determination record shows status=AUTO_APPROVED, confidence=0.9500000000000000, threshold=0.95. The audit log records the comparison as score >= threshold with both raw float64 values preserved without rounding. No manual review queue entry is created for this req…
Check
Pass / fail check

04

Eligibility Benefit Auth Requirement Determination

Mapped capabilities

57 scenarios

  • CPT/HCPCS-to-auth-requirement lookup

05

Physician Peer Review Workflow

Mapped capabilities

53 scenarios

  • Specialty Taxonomy Mapping

06

Request Intake Multi Channel Ingestion

Mapped capabilities

64 scenarios

  • Provider portal web-form happy-path submission

Frequently asked questions

What do the Corsac evals for Cohere Health test?+

Each eval pack tests Cohere Health's public product surface — including Clinical Criteria Medical Necessity Evaluation Engine, Clinical Documentation Capture Dtr Questionnaires, and Decisioning Auto Determination Output — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.

How are the Cohere Health evals scored?+

Pass/fail checks plus an LLM judge scoring 1–5 against each of the 349 Cohere Health cases — from Request Intake Multi Channel Ingestion (64 scenarios) down to the smallest pack — its own expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published to the Cohere Health library.

How many test cases does the Cohere Health library include?+

The Cohere Health eval library includes 349 graded test cases across 6 eval packs, the largest being Request Intake Multi Channel Ingestion with 64 scenarios. Each case defines an input, expected behavior, and pass/fail criteria.

How do I run these evals against Cohere Health or my own agent?+

Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run all 6 Cohere Health packs — Clinical Criteria Medical Necessity Evaluation Engine and Clinical Documentation Capture Dtr Questionnaires and the rest — against Cohere Health or your own agent with your own data.