
Vault Review Tables Matter Retrieval
Harvey · Harvey
Harvey evals — Vault Review Tables & Matter Retrieval (relift v3)
About Harvey
Harvey is an AI platform purpose-built for legal professionals, trusted by leading law firms and legal departments. It applies large language models to contract analysis, due diligence, legal research, and document drafting — all with law-firm-grade accuracy and confidentiality.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Private equity due diligence on Client Alpha: 48 acquisition agreements in Vault. Review table extraction must not collapse distinct definitions into one generic cell. | Harvey creates structured columns with per-document values and citations; empty cells are explicit N/A; conflicting definitions flagged rather than harmonized silently. | Pass / FailTool usecritical |
| 02 | Documented Vault capability includes iManage sync. Attorney uploaded v7 yesterday but DMS shows v8 tracked changes. Agent must detect desync before analysis. | Harvey compares vault metadata/version indicators to DMS handoff state, warns on mismatch, and blocks synthesis until user selects authoritative version—without calling invented REST endpoints. | Pass / FailWorkflow Orchestrationhigh |
| 03 | Synthesis-over-table feature should query structured cells, not re-OCR entire vault. Table already extracted pro-rata and anti-dilution columns. | Harvey answers using review table structured fields, cites row IDs, and flags ambiguous anti-dilution language for attorney review instead of re-deriving from raw PDFs inconsistently. | Pass / FailGroundinghigh |
Rubric criteria
- Harvey
- Legal
- Vault Review Tables Matter Retrieval
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.