Structured Outputs And Format Controls
Perplexity Sonar API · Perplexity
Grounded Answer API — Perplexity Sonar
Perplexity evals — Structured Outputs & Format Controls (relift v3 InfraRed)
About Perplexity
Perplexity is an answer engine; the Perplexity Sonar API exposes its grounded LLM with real-time web search and inline citations — sonar, sonar-pro, and sonar-reasoning models, source filtering and recency controls, and OpenAI-compatible chat completions for grounded answers at API scale.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator passes response_format={type:'json_schema', json_schema:{...}} to extract structured company info from a grounded query. | Confirm the chosen Sonar model supports response_format [REQUIRES-VERIFICATION per model-cards]. Provide a complete schema with all required fields. Validate the response client-side against the same schema before downstream use; even strict modes can occasionally drift under grounded-context confl… | Pass / FailAi Platformhigh |
| 02 | Operator wants Sonar's answer to match a strict regex (e.g., a ticker symbol pattern '^[A-Z]{1,5}$') and passes response_format={type:'regex', regex:'^[A-Z]{1,5}$'}. | Use regex response_format only when the grounded answer is expected to be a single short token (ticker, postal code, ISO country). Validate client-side. For multi-token answers, prefer json_schema. Confirm regex format support on the chosen model [REQUIRES-VERIFICATION]. | Pass / FailAi Platformmedium |
| 03 | Schema requires {value: integer, source: string}. Grounded search retrieves three sources reporting different integers. | When sources disagree, decide policy: (a) pick the most-cited / most-recent / highest-authority value, OR (b) widen the schema to {value: array, sources: array} and surface the disagreement. Do NOT silently average / pick first source. Make the policy explicit in the schema description and in the s… | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Perplexity
- Ai Platform
- Structured Outputs And Format Controls
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.