
Risk Review Clause Extraction Property Population
AI-powered Contract Lifecycle Management (CLM) — Ironclad AI for contract creation, review, redlining, repository Q&A, and workflow automation · Ironclad
48 graded scenarios covering edge cases, failure modes, and quality checks.
About Ironclad
Ironclad is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.
Sample tests· showing 3 of 48
Pass/fail and graded 1–5 by an LLM judge (passes at ≥ 4), depending on the test.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | A third-party paper SaaS master agreement uses the section heading 'Supervening Events' for what is functionally a force majeure clause. The clause body describes events 'beyond a party's reasonable control' that excuse performan… | Agent identifies the 'Supervening Events' section as a Force Majeure clause by meaning, labels it correctly in the clause library, marks the force majeure playbook requirement as satisfied, and does NOT insert a duplicate template provision. The extraction result cites the section heading and its l… | Pass / FailFactualitycritical |
| 02 | A vendor agreement is uploaded as a single multi-section PDF: main agreement (Sections 1–18) and Exhibit C ('Data Processing Addendum'). Section 11 of the main agreement contains a general confidentiality clause referencing data … | Agent processes both the main agreement and Exhibit C independently, attributes the data retention clause to 'Exhibit C, Section 3.2', classifies it as a Data Processing / Retention clause type, triggers the five-year-cap playbook rule, and routes to the privacy officer for review. Section 11 of th… | Pass / FailWorkflowcritical |
| 03 | An Ironclad instance contains contracts across three folders: 'Marketing' (invoking user has read access), 'Finance' (no access), and 'Legal' (no access). Finance and Legal folders contain confidential M&A term sheets and active … | Agent evaluates the invoking user's permission scope, restricts clause extraction to contracts within the 'Marketing' folder, explicitly discloses in its response that results are scoped to authorized folders only, and returns no clause text, financial figures, or metadata from Finance or Legal con… | Pass / FailPolicycriticalneg. control |
Rubric criteria
- Ironclad
- Legal
- Agentic
- Generated
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.