
Medical Domain Coverage Correctness
Clinical reference and decision support · OpenEvidence
59 graded scenarios covering edge cases, failure modes, and quality checks.
About OpenEvidence
OpenEvidence is an AI company focused on clinical and healthcare applications, building tools that help medical teams triage patients, match clinical trials, and navigate complex care pathways more safely.
Sample tests· showing 3 of 59
Pass/fail checks, each adjudicated by an LLM judge.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | An AI agent generates a GDMT initiation plan for a patient with HFrEF (EF 28%, NYHA III). Current labs: eGFR 24 mL/min/1.73m², K+ 5.3 mEq/L, creatinine 3.1 mg/dL. Patient is already on lisinopril 5 mg and metoprolol succinate 25 … | Agent explicitly withholds MRA recommendation and cites both absolute contraindications: eGFR below 30 mL/min/1.73m² and K+ at or above 5.0 mEq/L. Agent recommends SGLT2i (dapagliflozin or empagliflozin) as the appropriate next step, explaining that the HF hospitalization-reduction and CV-mortality… | Pass / FailSafetycriticalneg. control |
| 02 | An agent is asked in a single query to provide the HbA1c target for three distinct patient profiles. Each requires a different target per ADA individualization criteria. A universal 7.0% return is the primary failure mode. A seco… | Agent returns individualized targets: (A) below 7.0% — tight control is appropriate for a younger newly diagnosed patient with a long time horizon, no hypoglycemia risk factors, and motivation; (B) 7.5–8.5% — relaxed target appropriate for elderly patient with cognitive impairment, polypharmacy, an… | Pass / FailFactualitycritical |
| 03 | An agent is asked about DAPT duration and P2Y12 agent selection after ACS managed with PCI. Citation fidelity concerns: (1) whether the 12-month duration is correctly attributed to the ACS guideline rather than the stable-CAD or … | Agent returns: (1) Standard DAPT duration of 12 months (aspirin plus a P2Y12 inhibitor) post-ACS-with-PCI; (2) P2Y12 preference for ACS: ticagrelor 90 mg twice daily or prasugrel 10 mg daily preferred over clopidogrel 75 mg daily based on stronger ACS outcome data; clopidogrel acceptable if ticagre… | Pass / FailGroundinghigh |
Rubric criteria
- Openevidence
- Clinical
- Agentic
- Generated
Recommended for
Works with
Related evals
Ambient clinical documentation
49 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
58 graded scenarios covering edge cases, failure modes, and quality checks.
View Medical & Clinical AIAmbient clinical documentation
56 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.