Contract Review Negotiation Intelligence

How this eval is graded

Pass/fail checks, each adjudicated by an LLM judge.

Grade the agent's response against EACH criterion in expected.criteria independently (PASS/FAIL per criterion, using each criterion's match_criteria). The case passes only if EVERY criterion passes (all-pass) — partial completion fails. For negative criteria (is_negative=true), PASS means the agent did NOT exhibit the described behavior.

• C-001 · NEGPASS if every Texas-law enforceability conclusion is attributed solely to agreements whose governing-law clause specifies Texas; FAIL if any Texas non-compete enforceability rule or standard is applied to or cited in the analysis of a California-governed agreement.
• C-002PASS if the response correctly characterizes California non-compete provisions as void under Cal. Bus. & Prof. Code § 16600 (or equivalent authority) and explains the policy rationale for that voidness; FAIL if California non-compete law is omitted or described without reference to the statutory voidness default.
• C-003 · NEGPASS if the response provides separate, jurisdiction-specific enforceability assessments for Texas and California agreements; FAIL if the response assigns a single combined or blended enforceability score, rating, or conclusion that applies to both jurisdictions simultaneously.
• C-004 · NEGPASS if the comparison explicitly identifies the governing law for each agreement or comparison row (e.g., a dedicated 'Governing Law' column or per-row label); FAIL if governing law attribution is absent from the comparison output.
• C-005PASS if the output presents a structured side-by-side or tabular format that juxtaposes Texas and California agreement provisions row by row or column by column; FAIL if the response delivers only a narrative summary with no comparative structure distinguishing the two jurisdictions.
• C-006PASS if each jurisdiction row or section in the comparison contains at least one specific legal citation (statute, regulation, or case law); FAIL if any jurisdiction row presents an enforceability conclusion without a supporting citation.
• C-007 · NEGPASS if the response does not describe or imply that non-compete agreements are broadly or routinely enforceable under California law; FAIL if the response states, implies, or suggests that California courts enforce non-compete clauses without prominently acknowledging the statutory voidness default.
• C-008 · NEGPASS if enforceability conclusions are stated in jurisdiction-specific terms tied to Texas or California law; FAIL if the response uses generic parametric language such as 'generally enforceable,' 'typically enforced,' or 'commonly upheld' without qualifying the statement to a specific jurisdiction.
• C-009PASS if the Texas enforceability analysis references at least one Texas-specific enforceability requirement (e.g., reasonableness of duration, geographic scope, or legitimate business interest standard); FAIL if the Texas analysis contains no reference to Texas-specific enforceability elements or cites only generic common-law standards without Texas attribution.

Works with

Harvey

Frequently asked questions