Structured Outputs And Json Mode
GroqCloud API · Groq
Fast Inference — Groq (GroqCloud)
Groq evals — Structured Outputs & JSON Mode (relift v3 InfraRed)
About Groq
Groq builds the LPU (Language Processing Unit) inference engine and GroqCloud — an OpenAI-compatible API that serves leading open models (Llama, Mixtral, Gemma, Qwen) at very high tokens-per-second with low, deterministic latency. Developers use GroqCloud for real-time chat, tool use, structured outputs, and speech-to-text without managing GPU infrastructure.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Agent sets response_format={type:'json_object'} but the prompt never instructs the model to produce JSON, and the model returns prose. | When using response_format json_object, also instruct the model in the prompt to respond with JSON (the documented requirement). json_object guarantees syntactically valid JSON but not a particular shape — pair it with a clear schema description in the prompt. | Pass / FailAi Platformhigh |
| 02 | A schema constrains status to enum ['open','closed'] but the model returns 'in_progress'. | Enforce enum membership on validation; an out-of-enum value is a failed result that should be rejected and retried (or repaired), not mapped to a default silently. Surface the violation so the prompt or schema can be tightened. | Pass / FailAi Platformmedium |
| 03 | Agent needs guaranteed schema-valid output and sets response_format with a json_schema, but does not check whether the chosen model supports structured outputs. | Use response_format={type:'json_schema', json_schema:{...}} on a model that supports structured outputs, and confirm support against current docs — not every GroqCloud model honors strict json_schema. Validate the returned object against the schema client-side as a backstop. [REQUIRES-VERIFICATION]… | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Groq
- Ai Platform
- Structured Outputs And Json Mode
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.