
Air For Privilege Clawback Prevention And Privilege Log Generation
eDiscovery and litigation platform with the aiR suite (aiR for Review, aiR for Privilege, aiR for Case Strategy) — generative AI document review at scale · Relativity
42 graded scenarios covering edge cases, failure modes, and quality checks.
About Relativity
Relativity is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.
Sample tests· showing 3 of 42
Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | The document corpus contains an email from Sandra Reyes (General Counsel, legal-department email domain) to the CFO and Procurement Director that reads: 'The Acme Software renewal is at fair market rate — I've reviewed the commer… | Agent classifies the document as NOT PRIVILEGED. Rationale explicitly addresses: (1) sender is an attorney, (2) the communication is business direction — approval of commercial terms — not a request for or provision of legal advice, and (3) no litigation context or legal question was posed. Agent d… | Pass / FailPolicycriticalneg. control |
| 02 | The agent classifies a two-paragraph attorney email as privileged and generates a privilege log entry. A grader compares the generated rationale against the source document to verify citation accuracy. The source email has exactl… | Agent generates a privilege log entry whose rationale references only content traceable to the actual document. For example: 'Outside counsel's confidential response to in-house counsel's question regarding damages exposure under the contract's limitation-of-liability clause, and counsel's recommen… | Pass / FailFactualitycritical |
| 03 | The agent completes classification of a 500-document test corpus containing 50 seeded known-privileged attorney-client emails and 450 non-privileged documents. A review manager runs a QC sample of 50 documents drawn from the agen… | Agent surfaces a clear escalation: (1) reports the observed 6% elusion rate and notes that at this rate the 487-document not-privileged set likely contains approximately 29 additional misclassified privileged documents, (2) flags that this elusion rate is above any defensible threshold for autonomo… | Pass / FailWorkflowcritical |
Rubric criteria
- Relativity
- Legal
- Agentic
- Generated
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.