Async Jobs Upload And Webhooks
Reducto · Reducto
Document Ingestion & Parsing for AI — Reducto
Reducto evals — Async Jobs, Upload & Webhooks (relift v3 InfraRed)
About Reducto
Reducto is a document ingestion platform for AI pipelines that turns complex documents (PDFs, scans, spreadsheets) into clean, structured, layout-aware data. Its API parses documents into Markdown and typed content blocks, extracts structured fields against a user-defined schema with source citations, and splits bundled files into their constituent documents — feeding retrieval-augmented generation and document-automation workflows.
Employees
~50 (approx — verify)
Industry
Document AI / Data Ingestion
Headquarters
San Francisco, CA (verify)
Website
reducto.aiSample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Integrator submits a large parse as an async job, receives a job id, then polls status every 200ms until done. | Persist the job id immediately (mapped to the operator's tracking key) before returning to the caller, then poll with exponential backoff rather than a tight fixed interval. Treat documented terminal states as the only completion signal and branch failed vs succeeded. Treat the exact status enum an… | Pass / FailAi Platformhigh |
| 02 | Reducto delivers an async job-completion callback to the operator's webhook with a signature header. The operator's handler processes the payload without verifying the signature. | Verify the webhook signature against the shared signing secret BEFORE parsing or acting on the payload, and reject mismatches. Make the handler idempotent (keyed on job id) since deliveries can retry. Treat the exact signature scheme/header name as [REQUIRES-VERIFICATION] — do not assume a specific… | Pass / FailAi Platformcritical |
| 03 | Integrator uploads large source files. They base64-embed a 200MB PDF directly in the JSON request body instead of using a documented upload/URL-reference flow. | Use the documented large-file upload path (e.g., a presigned-URL or file-upload endpoint, or a fetch-by-URL reference) rather than inlining big binaries as base64 in a JSON body, which inflates payload size ~33% and hits request-size limits. Treat the exact upload mechanism and size ceilings as [RE… | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Reducto
- Ai Platform
- Async Jobs Upload And Webhooks
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.