
Deployments And Environments
Baseten · Baseten
AI Model Serving — Baseten
Evaluates Baseten's Deployments & Environments across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Model Serving eval coverage.
About Baseten
Baseten is a model serving platform that lets ML teams deploy, scale, and monitor any model — including custom fine-tunes and private weights — with production-grade autoscaling and GPU infrastructure. It supports both synchronous and asynchronous inference patterns.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator validates v2 of a model on /development/predict and clicks 'Promote to Production' in the workspace UI. | Promotion atomically points the production URL at the dev-tested image. In-flight production requests against the old image complete on their replicas; new requests are routed to the new image as replicas come up. Treat promotion as a release event and tag it in the operator's deploy log. | Pass / FailAi Platformhigh |
| 02 | v2 fails in production with elevated 5xx. Operator clicks 'Rollback' to v1 in the deployments UI. | Rollback re-routes production URL traffic to the prior image. v2 replicas drain their in-flight requests and terminate. Confirm rollback by observing replica counts and error rate restore. The rollback action does NOT delete v2 — keep it for forensics. | Pass / FailAi Platformcritical |
| 03 | Operator routes 10% of production traffic to v2 and 90% to v1 for 1 hour, then watches error-rate metrics. | Configure the canary split at the deployment level and watch per-version metrics (error rate, p95 latency, quality regressions) on a fixed observation window. Roll forward (100% v2) or roll back (100% v1) at the window end; do not let canary linger indefinitely. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Baseten
- Ai Platform
- Deployments And Environments
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Deployments And Environments eval for Baseten Baseten test?+
Evaluates Baseten's Deployments & Environments across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Model Serving eval coverage.
How is the Deployments And Environments eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Deployments And Environments pack for Baseten Baseten contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Deployments And Environments pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.