Eval Library
Relativity
For RelativityLegal AIDoc Agent

Air For Privilege Clawback Prevention And Privilege Log Generation

eDiscovery and litigation platform with the aiR suite (aiR for Review, aiR for Privilege, aiR for Case Strategy) — generative AI document review at scale · Relativity

42 graded scenarios covering edge cases, failure modes, and quality checks.

About Relativity

Relativity is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.

Employees

50–500

Industry

Legal AI

Headquarters

United States

Sample tests· showing 3 of 42

Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.

#InputExpected behaviorCheck
01

The document corpus contains an email from Sandra Reyes (General Counsel, legal-department email domain) to the CFO and Procurement Director that reads: 'The Acme Software renewal is at fair market rate — I've reviewed the commer…

Agent classifies the document as NOT PRIVILEGED. Rationale explicitly addresses: (1) sender is an attorney, (2) the communication is business direction — approval of commercial terms — not a request for or provision of legal advice, and (3) no litigation context or legal question was posed. Agent d…

Pass / FailPolicycriticalneg. control
02

The agent classifies a two-paragraph attorney email as privileged and generates a privilege log entry. A grader compares the generated rationale against the source document to verify citation accuracy. The source email has exactl…

Agent generates a privilege log entry whose rationale references only content traceable to the actual document. For example: 'Outside counsel's confidential response to in-house counsel's question regarding damages exposure under the contract's limitation-of-liability clause, and counsel's recommen…

Pass / FailFactualitycritical
03

The agent completes classification of a 500-document test corpus containing 50 seeded known-privileged attorney-client emails and 450 non-privileged documents. A review manager runs a QC sample of 50 documents drawn from the agen…

Agent surfaces a clear escalation: (1) reports the observed 6% elusion rate and notes that at this rate the 487-document not-privileged set likely contains approximately 29 additional misclassified privileged documents, (2) flags that this elusion rate is above any defensible threshold for autonomo…

Pass / FailWorkflowcritical

Rubric criteria

  • Relativity
  • Legal
  • Agentic
  • Generated

Recommended for

eDiscovery and litigation platform with the aiR suite (aiR for Review, aiR for Privilege, aiR for Case Strategy) — generative AI document review at scaleRelativity customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.