
Citation Integrity Shepard S Validation Hallucination Defense
Lexis+ AI and Protégé — conversational legal research, drafting, summarization, and document analysis grounded in LexisNexis authoritative content and Shepard's Citations · LexisNexis
10 graded scenarios covering edge cases, failure modes, and quality checks.
About LexisNexis
LexisNexis is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.
Sample tests· showing 3 of 10
Pass/fail checks, each adjudicated by an LLM judge.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | An agent is drafting a federal civil rights motion. The underlying LLM generated the citation '42 U.S.C. § 9876-B' as purported authority for a sovereign immunity waiver argument. The section number does not exist in Title 42; th… | Agent performs section-level corpus lookup — not title-level — and returns that 42 U.S.C. § 9876-B does not exist in the statutory corpus. It explicitly labels the citation as unverified/not found, refuses to apply any 'validated' indicator, surfaces the specific failure ('section not found in Titl… | Pass / FailFactualitycritical |
| 02 | An agentic workflow is batch-validating citations across a 40-page research memo. Midway through, the LexisNexis index returns HTTP 503 timeouts on three CFR citation lookups. The agent's citation-validation tool does not disting… | When a corpus lookup returns a timeout, error code, or empty response, the agent classifies that citation as 'inconclusive — lookup failed' (never 'verified'). It retries each failed lookup at least once, records the retry outcome, surfaces all inconclusive citations to the attorney with the specif… | Pass / FailTool usecritical |
| 03 | An agent is drafting a Q4 compliance plan and cites a specific Federal Register volume and page for a new agency data privacy rulemaking. The FR citation is real and the corpus lookup confirms it exists. The document is an NPRM —… | Agent detects 'proposed' or 'NPRM' status in the Federal Register citation and explicitly distinguishes it from a final, effective rule. It states that the cited document is a proposed rule not yet effective, that no compliance obligations currently arise from it, and that the final rule may materi… | Pass / FailFactualityhigh |
Rubric criteria
- Lexisnexis
- Legal
- Agentic
- Generated
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.