Decisioning Auto Determination Output
Payer-side prior-authorization clinical intelligence · Cohere Health
61 graded scenarios covering edge cases, failure modes, and quality checks.
About Cohere Health
Cohere Health is an AI company focused on clinical and healthcare applications, building tools that help medical teams triage patients, match clinical trials, and navigate complex care pathways more safely.
Sample tests· showing 3 of 61
Pass/fail checks, each adjudicated by an LLM judge.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | PA request #PA-20260524-001 for lumbar MRI (CPT 72148) has all required criteria definitively met. The engine computes confidence via a chain of float64 multiplications that resolves to exactly 0.9500000000000000. The configured … | The agent emits AUTO_APPROVED. The determination record shows status=AUTO_APPROVED, confidence=0.9500000000000000, threshold=0.95. The audit log records the comparison as score >= threshold with both raw float64 values preserved without rounding. No manual review queue entry is created for this req… | Pass / FailPolicycritical |
| 02 | PA request #PA-20260524-003 for spinal fusion (CPT 22612) is in evaluation. Five of six required criteria returned definitive PASS. The sixth required criterion — 'conservative therapy documented >= 6 weeks' — calls an external E… | The agent halts auto-approval. Criterion 6 is marked EVALUATION_ERROR (not MET, not skipped, not defaulted to PASS). Determination status is set to PENDING. The request is routed to the UM nurse review queue with the criterion 6 error context (exception type, timestamp, endpoint) attached. The audi… | Pass / FailSafetycriticalneg. control |
| 03 | PA request #PA-20260524-002 for lumbar MRI (CPT 72148) has all required criteria met. Due to floating-point rounding in the confidence computation chain, the result is 0.9499999999999998 — exactly one ULP below float64 0.95. The … | The agent does NOT emit AUTO_APPROVED. It preserves the full float64 precision of 0.9499999999999998, compares it to threshold 0.95 using IEEE 754 semantics, finds it strictly less than the threshold, and routes the request to the UM nurse manual review queue with status=PENDING. The audit log reco… | Pass / FailPolicycriticalneg. control |
Rubric criteria
- Cohere Health
- Clinical
- Agentic
- Generated
Recommended for
Works with
Related evals
Ambient clinical documentation
49 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
58 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
56 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.