Structured Outputs
Vercel AI SDK (`ai` npm package) · Vercel AI SDK
AI SDK — Vercel AI SDK
Vercel AI SDK evals — Structured Outputs (generateObject / streamObject) (relift v3 InfraRed)
About Vercel AI SDK
Vercel AI SDK is the open-source TypeScript-first AI framework from Vercel — the `ai` npm package. It gives developers provider-agnostic primitives (generateText, streamText, generateObject, streamObject), tool calling with Zod-typed schemas, AI SDK UI hooks (useChat, useCompletion, useObject) for React/Vue/Svelte, and RSC streaming via streamUI — so the same chat or agent code runs against OpenAI, Anthropic, Google, and more.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator calls generateObject with mode:'auto' against a provider that does not support JSON mode; output silently degrades to free-form text wrapped in {}. | mode:'auto' picks the best-supported strategy per provider (JSON mode where available, tool-call otherwise). For deterministic enforcement, pin mode:'json' (requires provider JSON-mode support) or mode:'tool' (uses a hidden tool whose parameters are the schema). Verify the chosen mode in provider d… | Pass / FailAi Platformhigh |
| 02 | generateObject with a Zod schema produces wrong-shape output more than 10% of the time. Operator pinned mode:'tool' but omitted schemaName and schemaDescription. | Set `schemaName: 'invoice'` and `schemaDescription: 'Structured invoice extracted from the user message'`. These names propagate to the underlying tool/JSON-mode call so the model has a concrete label — without them the SDK uses generic defaults that hurt accuracy. | Pass / FailAi Platformmedium |
| 03 | Operator wants a classifier returning one of 'urgent' | 'normal' | 'low'. | Use `output:'enum'` with `enum:['urgent','normal','low']`. result.object is typed as the union and the SDK validates the return against the list. Do not wrap a single-field enum in an object for this — enum output exists exactly for the classifier case. | Pass / FailAi Platformlow |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Vercel Ai Sdk
- Ai Platform
- Structured Outputs
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.