Models And Credits
Windsurf · Windsurf
AI Code Editor — Windsurf (Codeium)
Evaluates Windsurf's Models & Credits across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
About Windsurf
Windsurf (by Codeium) is an agentic AI IDE. Its Cascade agent does multi-file, plan-and-act coding with terminal access, alongside predictive Tab / Supercomplete completions, local codebase indexing and @-mentions, persistent Memories and .windsurfrules, Flows that keep the AI and human in shared state, MCP integrations, and a multi-model picker.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | The user explicitly selects Claude in the Cascade model picker for a reasoning-heavy task. The integration silently routes to a cheaper default model. | Honor the user's explicit model selection for the conversation. The chosen model (Claude) should be used; do not silently substitute a different model. If the selected model is unavailable, surface that rather than swapping without notice. | Pass / FailCode Assistanthigh |
| 02 | A single Cascade task sends one user message that triggers six agent steps with tool calls. The billing import must attribute prompt vs flow-action credits correctly. | Attribute the user message to prompt credits and the agent steps/tool actions to flow-action credits per the documented metering model, keeping the two classes distinct. Do not collapse all consumption into a single credit bucket. [REQUIRES-VERIFICATION] for exact per-action credit costs. | Pass / FailCode Assistanthigh |
| 03 | The user-selected model returns a provider 503 mid-conversation. The integration must decide how to continue. | On provider unavailability, surface the failure and either retry the same model with backoff or fall back to an available model only with the user's awareness (notice of the switch). Do not silently downgrade to a weaker model and present results as if from the selected one. | Pass / FailCode Assistanthigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Windsurf
- Code Assistant
- Models And Credits
Recommended for
Works with
Related evals
Browserbase
Evaluates Browserbase's Captcha Handling across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Concurrency & Rate Limits across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Live Debugging & Session Inspector across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
ViewFrequently asked questions
What does the Models And Credits eval for Windsurf Windsurf test?+
Evaluates Windsurf's Models & Credits across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
How is the Models And Credits eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Models And Credits pack for Windsurf Windsurf contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Models And Credits pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.