
Structured Outputs And Format Controls
Perplexity Sonar API · Perplexity
Grounded Answer API — Perplexity Sonar
Evaluates Perplexity's Structured Outputs & Format Controls across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Grounded Answer API eval coverage.
About Perplexity
Perplexity is an answer engine; the Perplexity Sonar API exposes its grounded LLM with real-time web search and inline citations — sonar, sonar-pro, and sonar-reasoning models, source filtering and recency controls, and OpenAI-compatible chat completions for grounded answers at API scale.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator passes response_format={type:'json_schema', json_schema:{...}} to extract structured company info from a grounded query. | Confirm the chosen Sonar model supports response_format [REQUIRES-VERIFICATION per model-cards]. Provide a complete schema with all required fields. Validate the response client-side against the same schema before downstream use; even strict modes can occasionally drift under grounded-context confl… | Pass / FailAi Platformhigh |
| 02 | Operator wants Sonar's answer to match a strict regex (e.g., a ticker symbol pattern '^[A-Z]{1,5}$') and passes response_format={type:'regex', regex:'^[A-Z]{1,5}$'}. | Use regex response_format only when the grounded answer is expected to be a single short token (ticker, postal code, ISO country). Validate client-side. For multi-token answers, prefer json_schema. Confirm regex format support on the chosen model [REQUIRES-VERIFICATION]. | Pass / FailAi Platformmedium |
| 03 | Schema requires {value: integer, source: string}. Grounded search retrieves three sources reporting different integers. | When sources disagree, decide policy: (a) pick the most-cited / most-recent / highest-authority value, OR (b) widen the schema to {value: array, sources: array} and surface the disagreement. Do NOT silently average / pick first source. Make the policy explicit in the schema description and in the s… | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Perplexity
- Ai Platform
- Structured Outputs And Format Controls
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Structured Outputs And Format Controls eval for Perplexity Perplexity Sonar API test?+
Evaluates Perplexity's Structured Outputs & Format Controls across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Grounded Answer API eval coverage.
How is the Structured Outputs And Format Controls eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Structured Outputs And Format Controls pack for Perplexity Perplexity Sonar API contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Structured Outputs And Format Controls pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.