Eval Library
Harvey
For HarveyLegal AIDoc AgentAnswer Relevance

Vault Review Tables Matter Retrieval

Harvey · Harvey

Harvey evals — Vault Review Tables & Matter Retrieval (relift v3)

About Harvey

Harvey is an AI platform purpose-built for legal professionals, trusted by leading law firms and legal departments. It applies large language models to contract analysis, due diligence, legal research, and document drafting — all with law-firm-grade accuracy and confidentiality.

Employees

~200

Industry

Legal AI

Headquarters

San Francisco, CA

Website

harvey.ai

Sample tests· showing 3 of 10

#InputExpected behaviorCheck
01

Private equity due diligence on Client Alpha: 48 acquisition agreements in Vault. Review table extraction must not collapse distinct definitions into one generic cell.

Harvey creates structured columns with per-document values and citations; empty cells are explicit N/A; conflicting definitions flagged rather than harmonized silently.

Pass / FailTool usecritical
02

Documented Vault capability includes iManage sync. Attorney uploaded v7 yesterday but DMS shows v8 tracked changes. Agent must detect desync before analysis.

Harvey compares vault metadata/version indicators to DMS handoff state, warns on mismatch, and blocks synthesis until user selects authoritative version—without calling invented REST endpoints.

Pass / FailWorkflow Orchestrationhigh
03

Synthesis-over-table feature should query structured cells, not re-OCR entire vault. Table already extracted pro-rata and anti-dilution columns.

Harvey answers using review table structured fields, cites row IDs, and flags ambiguous anti-dilution language for attorney review instead of re-deriving from raw PDFs inconsistently.

Pass / FailGroundinghigh

Rubric criteria

  • Harvey
  • Legal
  • Vault Review Tables Matter Retrieval

Recommended for

HarveyHarvey customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.