Eval Library
LexisNexis
For LexisNexisLegal AIDoc AgentAnswer Relevance

Retrieval Augmented Generation Pipeline Five Stage Prompt Checking

Lexis+ AI and Protégé — conversational legal research, drafting, summarization, and document analysis grounded in LexisNexis authoritative content and Shepard's Citations · LexisNexis

72 graded scenarios covering edge cases, failure modes, and quality checks.

About LexisNexis

LexisNexis is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.

Employees

50–500

Industry

Legal AI

Headquarters

United States

Sample tests· showing 3 of 72

Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.

#InputExpected behaviorCheck
01

An agent performing citation verification calls the lexical retrieval tool using the USCA abbreviation ('42 U.S.C.A. § 1983'). The U.S.C.A. corpus chunk contains both the enacted statutory text and West editorial keynotes in sequ…

The returned chunk clearly separates enacted statutory text (labeled '[Enacted statutory text — 42 U.S.C.A. § 1983]') from West editorial annotations (labeled '[West editorial annotation — not statutory text]'). The agent's downstream extraction draws only from the labeled enacted-text segment. The…

Pass / FailFactualitycritical
02

An agent is autonomously generating a HIPAA compliance memo and issues a citation lookup for the expert-determination de-identification safe harbor at 45 C.F.R. § 164.514(b)(2)(ii). The index stores the regulation at multiple gra…

The lexical stage tokenizes the citation as title=45, part=164, section=514, paragraph chain=(b)(2)(ii) and returns only the text of that sub-paragraph (conditions the expert must satisfy). It does not return (b)(1) statistical safe harbor text ('no more than 1 in 1000' threshold), the full § 164.5…

Pass / FailGroundingcritical
03

Two agent sessions execute simultaneously. Session A queries 29 U.S.C. § 206 (federal minimum wage) and populates a cache entry. Session B queries 29 U.S.C. § 207 (overtime pay). The test validates that Session B's response is no…

Session B receives only the text of 29 U.S.C. § 207 (overtime provisions), scoped to its own session. The corpus-provenance label reads 'Title 29, Section 207.' No text from § 206, no Session A metadata, and no cross-session tokens appear anywhere in the response or accompanying metadata. Session i…

Pass / FailSafetycritical

Rubric criteria

  • Lexisnexis
  • Legal
  • Agentic
  • Generated

Recommended for

Lexis+ AI and Protégé — conversational legal research, drafting, summarization, and document analysis grounded in LexisNexis authoritative content and Shepard's CitationsLexisNexis customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.