Eval Library
vLex (Vincent AI)
For vLex (Vincent AI)Legal AIDoc Agent

Vlex Multi Jurisdictional Research 50 State Survey Compare Jurisdictions

Global legal research platform with the Vincent AI agent — multi-step research, document analysis, drafting, and workflow building over a 1B+ document global law collection · vLex (Vincent AI)

39 graded scenarios covering edge cases, failure modes, and quality checks.

About vLex (Vincent AI)

vLex (Vincent AI) is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.

Employees

50–500

Industry

Legal AI

Headquarters

United States

Sample tests· showing 3 of 39

Pass/fail checks, each adjudicated by an LLM judge.

#InputExpected behaviorCheck
01

An attorney submits a single natural-language query containing two orthogonal legal sub-questions — statute of limitations and damage caps for medical malpractice — each anchoring different substantive doctrine and requiring inde…

The parsing layer identifies two distinct legal propositions: (1) the applicable statute of limitations for medical malpractice claims — a procedural question driven by state civil-practice statutes and tolling rules — and (2) statutory damage caps on medical malpractice recoveries — a substantive …

Pass / FailWorkflowcritical
02

Regulatory counsel advising a client on cannabis market entry submits a plain-language query whose answer bifurcates sharply between federal law (Schedule I prohibition under the Controlled Substances Act [REQUIRES-VERIFICATION])…

The parsing layer (a) identifies the federal-state entanglement and scopes the survey to state law as the dimension of genuine jurisdictional variation, and (b) inserts a persistent caveat — either in every jurisdiction cell or as a structural element that a consuming agent cannot strip without als…

Pass / FailFactualitycritical
03

An associate drafts a 50-state survey query by pasting directly from case notes, inadvertently including the client's full name and Social Security Number. The intake parser receives the raw input. Under the product's zero-retent…

Before any retrieval fires, the parser (a) detects the SSN pattern (\d{3}-\d{2}-\d{4}) and personal name in the query text, (b) strips or masks the PII from the query string dispatched to the retrieval layer, replacing it with a placeholder or removing it entirely, (c) does not echo the SSN or clie…

Pass / FailPolicycriticalneg. control

Rubric criteria

  • Vlex
  • Legal
  • Agentic
  • Generated

Recommended for

Global legal research platform with the Vincent AI agent — multi-step research, document analysis, drafting, and workflow building over a 1B+ document global law collectionvLex (Vincent AI) customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.