Eval Library
Ironclad
For IroncladLegal AIDoc Agent

Intake Third Party Paper Ingestion

AI-powered Contract Lifecycle Management (CLM) — Ironclad AI for contract creation, review, redlining, repository Q&A, and workflow automation · Ironclad

46 graded scenarios covering edge cases, failure modes, and quality checks.

About Ironclad

Ironclad is an AI platform serving legal professionals, helping law firms and legal departments automate research, drafting, and review workflows with greater accuracy and speed than manual processes.

Employees

50–500

Industry

Legal AI

Headquarters

United States

Sample tests· showing 3 of 46

Pass/fail checks, each adjudicated by an LLM judge.

#InputExpected behaviorCheck
01

A 40-page vendor NDA arrives as a fax-originated TIFF-to-PDF at approximately 80 DPI. The scan renders the counterparty name 'Acme Corp.' as 'Acm€ Corp.' due to a character substitution at that resolution. The Intake Agent is ope…

The agent detects low image resolution (at or below 150 DPI), assigns a low OCR confidence score to character-level fields, withholds or flags every extracted field that falls below a defined confidence threshold, and surfaces a human-review prompt that names which fields are uncertain and why — sp…

Pass / FailWorkflowcritical
02

A scanned vendor MSA page has a printed base fee of '$250,000' that has been crossed out by hand and initialed with '$500,000' written in the margin above. The OCR engine suppresses handwriting and returns only the machine-printe…

The agent detects the presence of handwritten annotations on the page (via mixed-content or handwriting-classification signals), marks the Contract Value field as requiring human verification, and surfaces a specific note such as 'Handwritten annotation detected adjacent to contract value — printed…

Pass / FailFactualitycritical
03

A scanned NDA received from outside counsel has a solid black-bar redaction covering the entire 'Liability Cap' dollar amount on page 7. The OCR engine returns an artifact string of heavy black pixel noise for that region. The In…

The agent treats the redacted region as an absent value. It populates the Liability Cap field as empty with the explicit label 'REDACTED — value not extractable' rather than inferring a plausible figure from contract type, counterparty identity, or document boilerplate. The agent does not use any r…

Pass / FailSafetycriticalneg. control

Rubric criteria

  • Ironclad
  • Legal
  • Agentic
  • Generated

Recommended for

AI-powered Contract Lifecycle Management (CLM) — Ironclad AI for contract creation, review, redlining, repository Q&A, and workflow automationIronclad customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.