Baseten
For BasetenAI Platform

Deployments And Environments

Baseten · Baseten

AI Model Serving — Baseten

Evaluates Baseten's Deployments & Environments across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Model Serving eval coverage.

About Baseten

Baseten is a model serving platform that lets ML teams deploy, scale, and monitor any model — including custom fine-tunes and private weights — with production-grade autoscaling and GPU infrastructure. It supports both synchronous and asynchronous inference patterns.

Employees

~100

Industry

Model Serving

Headquarters

San Francisco, CA

Website

baseten.co

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Operator validates v2 of a model on /development/predict and clicks 'Promote to Production' in the workspace UI.

Promotion atomically points the production URL at the dev-tested image. In-flight production requests against the old image complete on their replicas; new requests are routed to the new image as replicas come up. Treat promotion as a release event and tag it in the operator's deploy log.

Pass / FailAi Platformhigh
02

v2 fails in production with elevated 5xx. Operator clicks 'Rollback' to v1 in the deployments UI.

Rollback re-routes production URL traffic to the prior image. v2 replicas drain their in-flight requests and terminate. Confirm rollback by observing replica counts and error rate restore. The rollback action does NOT delete v2 — keep it for forensics.

Pass / FailAi Platformcritical
03

Operator routes 10% of production traffic to v2 and 90% to v1 for 1 hour, then watches error-rate metrics.

Configure the canary split at the deployment level and watch per-version metrics (error rate, p95 latency, quality regressions) on a fixed observation window. Roll forward (100% v2) or roll back (100% v1) at the window end; do not let canary linger indefinitely.

Pass / FailAi Platformhigh

Unlock full benchmark

6 more test cases

Use this benchmark

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Baseten
  • Ai Platform
  • Deployments And Environments

Recommended for

BasetenBaseten customers

Works with

Related evals

Frequently asked questions

What does the Deployments And Environments eval for Baseten Baseten test?+

Evaluates Baseten's Deployments & Environments across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Model Serving eval coverage.

How is the Deployments And Environments eval scored?+

The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

How many test cases does this eval pack include?+

The Deployments And Environments pack for Baseten Baseten contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.

How do I run this eval?+

Sign up for Corsac, connect your model or agent endpoint, and run the Deployments And Environments pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.