Eval directory · AI Platform

Evals for Reducto

Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Reducto AI products.

About Reducto

Reducto is a document-processing platform for classifying, parsing, extracting, splitting, and editing documents through APIs and workflows.

Industry

Document AI

Website

reducto.ai

Use the eval library for Reducto

All 73 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.

Related in AI Platform

All evals →

More AI Platform eval libraries

Coverage map

What would you measure for Reducto?

8 areas · 73 graded scenarios

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Async Jobs Upload And Webhooks

Evaluates Reducto's Async Jobs, Upload & Webhooks across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

async job submit + poll lifecycle
webhook signature verification
presigned / direct file upload

Public sample case

Input: Integrator submits a large parse as an async job, receives a job id, then polls status every 200ms until done.
Expected behavior: Persist the job id immediately (mapped to the operator's tracking key) before returning to the caller, then poll with exponential backoff rather than a tight fixed interval. Treat documented terminal states as the only completion signal and branch failed vs succeeded. Treat the exact status enum an…
Check: Pass / fail check

Auth Privacy And Governance

Evaluates Reducto's Auth, Privacy & Governance across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

10 scenarios

API key handling and scope
PII in documents and retention
data-deletion / GDPR subject request

Public sample case

Input: Integrator wires the Reducto API key into a browser-side script so the frontend can call the parse API directly.
Expected behavior: Keep the API key server-side only; never ship it in a browser bundle or mobile app where it can be extracted. Front document submission through the operator's backend which holds the key. Scope and rotate keys per the documented key model. Treat fine-grained key scoping support as [REQUIRES-VERIFIC…
Check: Pass / fail check

Chunking For Embeddings

Evaluates Reducto's Chunking for Embeddings across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

layout-aware vs fixed-size chunking
chunk metadata / provenance
table chunking strategy

Public sample case

Input: The integrator chunks parsed output by a fixed 1,000-character window, slicing mid-sentence and mid-table, instead of using Reducto's layout-aware block boundaries.
Expected behavior: Chunk on semantic/layout boundaries (section, paragraph, table) surfaced by the parse output rather than a blind character window that splits sentences, tables, and list items. Layout-aware chunks improve retrieval precision. Where a hard size cap is needed, split at the nearest block boundary, not…
Check: Pass / fail check

Extract Schema And Citations

Evaluates Reducto's Extract (Schema-driven + Citations) across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

extract schema definition
citation / source-grounding usage
missing-field / null handling

Ocr And Layout Fidelity

Evaluates Reducto's OCR & Layout Fidelity across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

character-level OCR accuracy gating
bounding-box / coordinate usage
multilingual / non-Latin script

Parse Document To Structured

Evaluates Reducto's Parse (Document to Structured) across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

parse response block structure
parse output mode selection
multi-page reading order

Split Classification And Segmentation

Evaluates Reducto's Split (Classification & Segmentation) across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

split a bundled packet
classification label set
segment boundary off-by-one

Tables And Figures

Evaluates Reducto's Tables & Figures across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Document Ingestion & Parsing for AI eval coverage.

Mapped capabilities

9 scenarios

table cell structure preservation
merged / spanning cells
multi-page / continued tables

Frequently asked questions

What do the Corsac evals for Reducto test?+

Each eval pack tests Reducto's public product surface — including Async Jobs Upload And Webhooks, Auth Privacy And Governance, and Chunking For Embeddings — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.

How are the Reducto evals scored?+

Pass/fail checks plus an LLM judge scoring 1–5 against each of the 73 Reducto cases — from Auth Privacy And Governance (10 scenarios) down to the smallest pack — its own expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published to the Reducto library.

How many test cases does the Reducto library include?+

The Reducto eval library includes 73 graded test cases across 8 eval packs, the largest being Auth Privacy And Governance with 10 scenarios. Each case defines an input, expected behavior, and pass/fail criteria.

How do I run these evals against Reducto or my own agent?+

Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run all 8 Reducto packs — Async Jobs Upload And Webhooks and Auth Privacy And Governance and the rest — against Reducto or your own agent with your own data.