Eval Library
Harvey
For HarveyLegal AIDoc AgentAnswer Relevance

Assistant Citation Grounded Qa

Harvey · Harvey

Harvey evals — Assistant Citation-Grounded Q&A (relift v3)

About Harvey

Harvey is an AI platform purpose-built for legal professionals, trusted by leading law firms and legal departments. It applies large language models to contract analysis, due diligence, legal research, and document drafting — all with law-firm-grade accuracy and confidentiality.

Employees

~200

Industry

Legal AI

Headquarters

San Francisco, CA

Website

harvey.ai

Sample tests· showing 3 of 10

#InputExpected behaviorCheck
01

Associate on Matter REF-2041 for Client Alpha needs a litigation memo paragraph on forum selection. Harvey Assistant returns inline citations to three Vault contracts. A downstream agent will paste citations into a court filing w…

Every legal proposition is tied to a retrieved Vault passage the user can open; if no passage supports the proposition, Harvey states insufficient Vault support rather than inventing a citation or extrapolating beyond the quoted text.

Pass / FailGroundingcritical
02

Vault for Matter REF-3310 contains no Delaware veil-piercing cases. The model has parametric knowledge of typical Delaware precedents. The associate will treat any citation block as Vault-verified.

Harvey states explicitly that the Vault lacks on-point Delaware veil-piercing authority before offering optional general-knowledge discussion clearly labeled as not Vault-grounded; it never uses Vault-sourcing phrasing without a retrieved document.

Pass / FailFactualitycritical
03

One compound Assistant prompt spans three orthogonal clause families that rarely co-occur in a single document. Agentic search must decompose queries or miss schedules attached separately.

Harvey emits at least three distinct retrieval queries—one targeting IP indemnification, one limitation-of-liability carve-outs, one GDPR/Article 28 processor language—without bundling all three topics into a single omnibus query string.

Pass / FailTool usehigh

Rubric criteria

  • Harvey
  • Legal
  • Assistant Citation Grounded Qa

Recommended for

HarveyHarvey customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.