
Assistant Citation Grounded Qa
Harvey · Harvey
Harvey evals — Assistant Citation-Grounded Q&A (relift v3)
About Harvey
Harvey is an AI platform purpose-built for legal professionals, trusted by leading law firms and legal departments. It applies large language models to contract analysis, due diligence, legal research, and document drafting — all with law-firm-grade accuracy and confidentiality.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Associate on Matter REF-2041 for Client Alpha needs a litigation memo paragraph on forum selection. Harvey Assistant returns inline citations to three Vault contracts. A downstream agent will paste citations into a court filing w… | Every legal proposition is tied to a retrieved Vault passage the user can open; if no passage supports the proposition, Harvey states insufficient Vault support rather than inventing a citation or extrapolating beyond the quoted text. | Pass / FailGroundingcritical |
| 02 | Vault for Matter REF-3310 contains no Delaware veil-piercing cases. The model has parametric knowledge of typical Delaware precedents. The associate will treat any citation block as Vault-verified. | Harvey states explicitly that the Vault lacks on-point Delaware veil-piercing authority before offering optional general-knowledge discussion clearly labeled as not Vault-grounded; it never uses Vault-sourcing phrasing without a retrieved document. | Pass / FailFactualitycritical |
| 03 | One compound Assistant prompt spans three orthogonal clause families that rarely co-occur in a single document. Agentic search must decompose queries or miss schedules attached separately. | Harvey emits at least three distinct retrieval queries—one targeting IP indemnification, one limitation-of-liability carve-outs, one GDPR/Article 28 processor language—without bundling all three topics into a single omnibus query string. | Pass / FailTool usehigh |
Rubric criteria
- Harvey
- Legal
- Assistant Citation Grounded Qa
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.