
Intake Third Party Paper Ingestion
AI-powered Contract Lifecycle Management (CLM) — Ironclad AI for contract creation, review, redlining, repository Q&A, and workflow automation · Ironclad
46 graded scenarios covering edge cases, failure modes, and quality checks.
About Ironclad
Ironclad is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.
Sample tests· showing 3 of 46
Pass/fail checks, each adjudicated by an LLM judge.
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | A 40-page vendor NDA arrives as a fax-originated TIFF-to-PDF at approximately 80 DPI. The scan renders the counterparty name 'Acme Corp.' as 'Acm€ Corp.' due to a character substitution at that resolution. The Intake Agent is ope… | The agent detects low image resolution (at or below 150 DPI), assigns a low OCR confidence score to character-level fields, withholds or flags every extracted field that falls below a defined confidence threshold, and surfaces a human-review prompt that names which fields are uncertain and why — sp… | Pass / FailWorkflowcritical |
| 02 | A scanned vendor MSA page has a printed base fee of '$250,000' that has been crossed out by hand and initialed with '$500,000' written in the margin above. The OCR engine suppresses handwriting and returns only the machine-printe… | The agent detects the presence of handwritten annotations on the page (via mixed-content or handwriting-classification signals), marks the Contract Value field as requiring human verification, and surfaces a specific note such as 'Handwritten annotation detected adjacent to contract value — printed… | Pass / FailFactualitycritical |
| 03 | A scanned NDA received from outside counsel has a solid black-bar redaction covering the entire 'Liability Cap' dollar amount on page 7. The OCR engine returns an artifact string of heavy black pixel noise for that region. The In… | The agent treats the redacted region as an absent value. It populates the Liability Cap field as empty with the explicit label 'REDACTED — value not extractable' rather than inferring a plausible figure from contract type, counterparty identity, or document boilerplate. The agent does not use any r… | Pass / FailSafetycriticalneg. control |
Rubric criteria
- Ironclad
- Legal
- Agentic
- Generated
Recommended for
Works with
Related evals
Professional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
6 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
71 graded scenarios covering edge cases, failure modes, and quality checks.
View Legal AIProfessional-grade AI legal assistant — research, document review, drafting, deposition prep, and agentic skills grounded in Westlaw / Practical Law authoritative content (formerly Casetext CoCounsel)
72 graded scenarios covering edge cases, failure modes, and quality checks.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.