Eval Library
P
For PerplexityAI Platform

Structured Outputs And Format Controls

Perplexity Sonar API · Perplexity

Grounded Answer API — Perplexity Sonar

Perplexity evals — Structured Outputs & Format Controls (relift v3 InfraRed)

About Perplexity

Perplexity is an answer engine; the Perplexity Sonar API exposes its grounded LLM with real-time web search and inline citations — sonar, sonar-pro, and sonar-reasoning models, source filtering and recency controls, and OpenAI-compatible chat completions for grounded answers at API scale.

Employees

~200

Industry

Search / Answer API

Headquarters

San Francisco, CA

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Operator passes response_format={type:'json_schema', json_schema:{...}} to extract structured company info from a grounded query.

Confirm the chosen Sonar model supports response_format [REQUIRES-VERIFICATION per model-cards]. Provide a complete schema with all required fields. Validate the response client-side against the same schema before downstream use; even strict modes can occasionally drift under grounded-context confl…

Pass / FailAi Platformhigh
02

Operator wants Sonar's answer to match a strict regex (e.g., a ticker symbol pattern '^[A-Z]{1,5}$') and passes response_format={type:'regex', regex:'^[A-Z]{1,5}$'}.

Use regex response_format only when the grounded answer is expected to be a single short token (ticker, postal code, ISO country). Validate client-side. For multi-token answers, prefer json_schema. Confirm regex format support on the chosen model [REQUIRES-VERIFICATION].

Pass / FailAi Platformmedium
03

Schema requires {value: integer, source: string}. Grounded search retrieves three sources reporting different integers.

When sources disagree, decide policy: (a) pick the most-cited / most-recent / highest-authority value, OR (b) widen the schema to {value: array, sources: array} and surface the disagreement. Do NOT silently average / pick first source. Make the policy explicit in the schema description and in the s…

Pass / FailAi Platformcritical

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Perplexity
  • Ai Platform
  • Structured Outputs And Format Controls

Recommended for

Perplexity Sonar APIPerplexity customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.