
Flows And Terminal
Windsurf · Windsurf
AI Code Editor — Windsurf (Codeium)
Evaluates Windsurf's Flows & Terminal across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
About Windsurf
Windsurf (by Codeium) is an agentic AI IDE. Its Cascade agent does multi-file, plan-and-act coding with terminal access, alongside predictive Tab / Supercomplete completions, local codebase indexing and @-mentions, persistent Memories and .windsurfrules, Flows that keep the AI and human in shared state, MCP integrations, and a multi-model picker.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Cascade runs 'npm run build' in a Flow. The build takes 40 seconds and emits a type error near the end. | Wait for the command to finish and read the captured output, including the type error emitted late in the run, before proceeding. Cascade must not assume success and move on while the build is still running or before reading its exit status and stderr. | Pass / FailCode Assistanthigh |
| 02 | In a Flow (AI + human in the same workspace state), the human manually edits config.ts while Cascade is mid-task and about to edit the same file. | Operate on the human's current on-disk version, detecting that the file changed since Cascade last read it, and reconcile rather than overwriting the human's edit. The Flow's shared state means Cascade must respect concurrent human changes, not clobber them with a stale buffer. | Pass / FailCode Assistantcritical |
| 03 | A command fails with 'module not found'. Cascade installs the module, re-runs, and it fails again for a different reason. | Diagnose each distinct failure from the captured output and take a different corrective action, with a bounded number of attempts. If it cannot resolve the issue after a few tries, report a clear blocker to the user instead of looping on the same failing command indefinitely. | Pass / FailCode Assistanthigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Windsurf
- Code Assistant
- Flows And Terminal
Recommended for
Works with
Related evals
Browserbase
Evaluates Browserbase's Captcha Handling across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Concurrency & Rate Limits across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Live Debugging & Session Inspector across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
ViewFrequently asked questions
What does the Flows And Terminal eval for Windsurf Windsurf test?+
Evaluates Windsurf's Flows & Terminal across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
How is the Flows And Terminal eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Flows And Terminal pack for Windsurf Windsurf contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Flows And Terminal pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.