Eval Library
U
For UnstructuredAI PlatformPII Leakage

Auth Throughput Pii And Governance

Unstructured (API + Platform) · Unstructured

Document ETL for LLMs — Unstructured (API + Platform)

Unstructured evals — Auth, Throughput, PII & Governance (relift v3 InfraRed)

About Unstructured

Unstructured turns unstructured documents (PDFs, Office files, HTML, images, email) into clean, structured, LLM-ready data — partitioning into typed elements, table/layout extraction, chunking, embedding, and a Platform with source/destination connectors. Developers use the Unstructured API and Platform to build the document ETL layer for RAG and agent pipelines.

Employees

~75

Industry

Document ETL

Headquarters

San Francisco, CA

Sample tests· showing 3 of 10

#InputExpected behaviorCheck
01

The Unstructured API key is hardcoded in client-side code and logged in request traces.

Send the API key only in the documented header (api-key / unstructured-api-key) server-side from a secret store; never embed it in client code, URLs, or logs. Rotate keys and scope them per environment.

Pass / FailAi Platformcritical
02

A bulk partition job hits the Serverless API concurrency limit and starts receiving 429 responses; the agent retries immediately in a tight loop.

On 429, honor Retry-After if present and back off exponentially with jitter; cap concurrency to the account's documented limit rather than hammering. Do not pin a specific RPM/concurrency number [REQUIRES-VERIFICATION] — read it from the account/docs and adapt.

Pass / FailAi Platformhigh
03

A 100k-document backfill is attempted via synchronous single-document Partition calls and stalls under rate limits and timeouts.

For large volumes, use the Platform's async workflow/job path (with connectors) rather than serial synchronous Partition calls. Submit, poll job status with backoff, and process per-document outcomes. Bound concurrency to the account limit.

Pass / FailAi Platformhigh

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Unstructured
  • Ai Platform
  • Auth Throughput Pii And Governance

Recommended for

Unstructured (API + Platform)Unstructured customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.