Mistral Json Mode And Structured Output
Mistral AI API · Mistral AI
Foundation Model & API — Mistral AI
Mistral AI evals — JSON Mode & Structured Output (relift v3 InfraRed)
About Mistral AI
Mistral AI is a European foundation-model company offering open-weight and commercial models (Mistral Large, Codestral, Pixtral) via La Plateforme, plus Le Chat, embeddings, fine-tuning, and agents — with a strong emphasis on EU data residency.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator sets response_format={type:'json_object'} and expects field-level schema guarantees for a downstream DB write. | json_object guarantees syntactically valid JSON, not your schema; use response_format json_schema for field guarantees, or post-validate every field. Choose based on whether structure must be enforced. | Pass / FailAi Platformhigh |
| 02 | Operator enables json_object but the system/user prompt never asks for JSON, and outputs are sometimes empty or whitespace. | When using json_object, also instruct the model in the prompt to produce JSON; the format flag constrains syntax but the prompt must still request the structure. Validate non-empty output. | Pass / FailAi Platformmedium |
| 03 | The mistralai SDK offers a typed parse path; the team hand-rolls json.loads and loses type validation. | Prefer the SDK's structured/typed parse where available so type mismatches surface as errors; hand-rolled json.loads silently accepts wrong-typed fields. Validate types at the boundary regardless. | Pass / FailAi Platformlow |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mistral
- Ai Platform
- Json Mode And Structured Output
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.