Perplexity
For PerplexityAI Platform

Structured Outputs And Format Controls

Perplexity Sonar API · Perplexity

Grounded Answer API — Perplexity Sonar

Evaluates Perplexity's Structured Outputs & Format Controls across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Grounded Answer API eval coverage.

About Perplexity

Perplexity is an answer engine; the Perplexity Sonar API exposes its grounded LLM with real-time web search and inline citations — sonar, sonar-pro, and sonar-reasoning models, source filtering and recency controls, and OpenAI-compatible chat completions for grounded answers at API scale.

Employees

~200

Industry

Search / Answer API

Headquarters

San Francisco, CA

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Operator passes response_format={type:'json_schema', json_schema:{...}} to extract structured company info from a grounded query.

Confirm the chosen Sonar model supports response_format [REQUIRES-VERIFICATION per model-cards]. Provide a complete schema with all required fields. Validate the response client-side against the same schema before downstream use; even strict modes can occasionally drift under grounded-context confl…

Pass / FailAi Platformhigh
02

Operator wants Sonar's answer to match a strict regex (e.g., a ticker symbol pattern '^[A-Z]{1,5}$') and passes response_format={type:'regex', regex:'^[A-Z]{1,5}$'}.

Use regex response_format only when the grounded answer is expected to be a single short token (ticker, postal code, ISO country). Validate client-side. For multi-token answers, prefer json_schema. Confirm regex format support on the chosen model [REQUIRES-VERIFICATION].

Pass / FailAi Platformmedium
03

Schema requires {value: integer, source: string}. Grounded search retrieves three sources reporting different integers.

When sources disagree, decide policy: (a) pick the most-cited / most-recent / highest-authority value, OR (b) widen the schema to {value: array, sources: array} and surface the disagreement. Do NOT silently average / pick first source. Make the policy explicit in the schema description and in the s…

Pass / FailAi Platformcritical

Unlock full benchmark

6 more test cases

Use this benchmark

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Perplexity
  • Ai Platform
  • Structured Outputs And Format Controls

Recommended for

Perplexity Sonar APIPerplexity customers

Works with

Related evals

Frequently asked questions

What does the Structured Outputs And Format Controls eval for Perplexity Perplexity Sonar API test?+

Evaluates Perplexity's Structured Outputs & Format Controls across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Grounded Answer API eval coverage.

How is the Structured Outputs And Format Controls eval scored?+

The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

How many test cases does this eval pack include?+

The Structured Outputs And Format Controls pack for Perplexity Perplexity Sonar API contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.

How do I run this eval?+

Sign up for Corsac, connect your model or agent endpoint, and run the Structured Outputs And Format Controls pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.