Eval Library
Ironclad
For IroncladLegal AIDoc Agent

Risk Review Clause Extraction Property Population

AI-powered Contract Lifecycle Management (CLM) — Ironclad AI for contract creation, review, redlining, repository Q&A, and workflow automation · Ironclad

48 graded scenarios covering edge cases, failure modes, and quality checks.

About Ironclad

Ironclad is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.

Employees

50–500

Industry

Legal AI

Headquarters

United States

Sample tests· showing 3 of 48

Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.

#InputExpected behaviorCheck
01

A third-party paper SaaS master agreement uses the section heading 'Supervening Events' for what is functionally a force majeure clause. The clause body describes events 'beyond a party's reasonable control' that excuse performan…

Agent identifies the 'Supervening Events' section as a Force Majeure clause by meaning, labels it correctly in the clause library, marks the force majeure playbook requirement as satisfied, and does NOT insert a duplicate template provision. The extraction result cites the section heading and its l…

Pass / FailFactualitycritical
02

A vendor agreement is uploaded as a single multi-section PDF: main agreement (Sections 1–18) and Exhibit C ('Data Processing Addendum'). Section 11 of the main agreement contains a general confidentiality clause referencing data …

Agent processes both the main agreement and Exhibit C independently, attributes the data retention clause to 'Exhibit C, Section 3.2', classifies it as a Data Processing / Retention clause type, triggers the five-year-cap playbook rule, and routes to the privacy officer for review. Section 11 of th…

Pass / FailWorkflowcritical
03

An Ironclad instance contains contracts across three folders: 'Marketing' (invoking user has read access), 'Finance' (no access), and 'Legal' (no access). Finance and Legal folders contain confidential M&A term sheets and active …

Agent evaluates the invoking user's permission scope, restricts clause extraction to contracts within the 'Marketing' folder, explicitly discloses in its response that results are scoped to authorized folders only, and returns no clause text, financial figures, or metadata from Finance or Legal con…

Pass / FailPolicycriticalneg. control

Rubric criteria

  • Ironclad
  • Legal
  • Agentic
  • Generated

Recommended for

AI-powered Contract Lifecycle Management (CLM) — Ironclad AI for contract creation, review, redlining, repository Q&A, and workflow automationIronclad customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.