Mistral Embeddings And Retrieval
Mistral AI API · Mistral AI
Foundation Model & API — Mistral AI
Mistral AI evals — Embeddings & Retrieval (relift v3 InfraRed)
About Mistral AI
Mistral AI is a European foundation-model company offering open-weight and commercial models (Mistral Large, Codestral, Pixtral) via La Plateforme, plus Le Chat, embeddings, fine-tuning, and agents — with a strong emphasis on EU data residency.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Team builds a retrieval index with mistral-embed but later embeds new queries with a different embedding model, then compares vectors. | All vectors in an index must come from the same embedding model; query vectors must be produced by mistral-embed if the index was built with mistral-embed. Mixing models makes similarity meaningless. | Pass / FailAi Platformcritical |
| 02 | Engineer hard-codes a vector dimension for mistral-embed copied from a forum post and provisions the vector store to that size. | Read the embedding dimensionality from an actual mistral-embed response and provision the store to match; do not hard-code a number from an unofficial source. Mark any cited dimension [REQUIRES-VERIFICATION] until confirmed against a live response. | Pass / FailAi Platformhigh |
| 03 | Retrieval ranks by raw dot product over embeddings without confirming normalization and gets inconsistent ordering versus cosine. | Confirm whether mistral-embed vectors are unit-normalized; if not guaranteed, normalize consistently before using cosine, and ensure the vector store's distance metric matches. Treat the normalization assumption as [REQUIRES-VERIFICATION]. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mistral
- Ai Platform
- Embeddings And Retrieval
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.