
Mistral Chat Completions And Streaming
Mistral AI API · Mistral AI
Foundation Model & API — Mistral AI
Evaluates Mistral AI's Chat Completions & Streaming across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
About Mistral AI
Mistral AI is a European foundation-model company offering open-weight and commercial models (Mistral Large, Codestral, Pixtral) via La Plateforme, plus Le Chat, embeddings, fine-tuning, and agents — with a strong emphasis on EU data residency.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Agent streams /v1/chat/completions with stream=true and max_tokens=256 for a long answer; the final SSE chunk reports finish_reason='length'. | Detect finish_reason='length' on the terminal chunk and treat the answer as truncated — surface a partial-completion to the caller or continue by appending the assistant text and re-requesting. Never present truncated output as complete. | Pass / FailAi Platformcritical |
| 02 | Agent consumes the SSE stream from /v1/chat/completions and stops reading at the first empty data line instead of the terminal 'data: [DONE]' marker. | Parse the SSE stream until the explicit 'data: [DONE]' sentinel; accumulate delta.content across chunks and only finalize on the terminator. Do not treat keep-alive or blank lines as end-of-stream. | Pass / FailAi Platformhigh |
| 03 | Operator sets stop=['\n\nUser:'] expecting generation to halt before the next turn marker; the returned text still contains the marker. | Configure stop sequences correctly and verify finish_reason='stop'; the stop string itself is not included in the output, so do not also strip it manually and corrupt the text. Confirm the marker matches exactly. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mistral
- Ai Platform
- Chat Completions And Streaming
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Mistral Chat Completions And Streaming eval for Mistral AI Mistral AI API test?+
Evaluates Mistral AI's Chat Completions & Streaming across 10 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
How is the Mistral Chat Completions And Streaming eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Mistral Chat Completions And Streaming pack for Mistral AI Mistral AI API contains 10 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Mistral Chat Completions And Streaming pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.