
Assistant Agentic Search And Iterative Source Expansion
Agentic legal work platform — Assistant, Vault, and Workflows for research, drafting, document review, and multi-step matter execution across 24+ practice areas · Harvey
18 graded scenarios covering edge cases, failure modes, and quality checks.
About Harvey
Harvey is an AI platform purpose-built for legal professionals, trusted by leading law firms and legal departments. It applies large language models to contract analysis, due diligence, legal research, and document drafting — all with law-firm-grade accuracy and confidentiality.
Sample tests· showing 3 of 18
Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Associate submits a single compound prompt spanning three legally distinct issues: IP indemnification obligations, limitation-of-liability carve-outs, and GDPR data processor requirements. A monolithic query would retrieve only d… | Agent emits at least three distinct, non-overlapping queries: one targeting IP indemnification clauses (e.g., 'IP indemnification hold harmless vendor agreement'), one targeting limitation-of-liability and consequential damage carve-out language (e.g., 'limitation of liability consequential damages… | Pass / FailTool usecritical |
| 02 | Partner asks Harvey to research what protections German employees have when terminated without cause. The vault contains German employment law resources and German-language employment agreements. If German-specific terms yield sp… | Agent formulates at least one query containing 'Kündigungsschutz' or 'Kündigungsschutzgesetz' or 'KSchG,' and at least one query containing 'ordentliche Kündigung' or 'Betriebsrat' (works council consent). Agent does not substitute 'at-will employment,' 'wrongful termination,' or the U.S. WARN Act … | Pass / FailGroundingcritical |
| 03 | Attorney explicitly restricts search to executed contracts in the firm's vault. Harvey's iterative expansion logic may, upon finding sparse results, silently route follow-up queries to external legal databases or web search. The … | All queries — initial and every iterative expansion round — are routed exclusively to the vault's executed contracts corpus. If initial results are sparse, agent surfaces that finding with a statement explicitly referencing the vault-only constraint, and waits for user direction before expanding sc… | Pass / FailPolicycritical |
Rubric criteria
- Harvey
- Legal
- Agentic
- Generated
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.