Windsurf
For WindsurfCode Assistant

Flows And Terminal

Windsurf · Windsurf

AI Code Editor — Windsurf (Codeium)

Evaluates Windsurf's Flows & Terminal across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.

About Windsurf

Windsurf (by Codeium) is an agentic AI IDE. Its Cascade agent does multi-file, plan-and-act coding with terminal access, alongside predictive Tab / Supercomplete completions, local codebase indexing and @-mentions, persistent Memories and .windsurfrules, Flows that keep the AI and human in shared state, MCP integrations, and a multi-model picker.

Employees

~200

Industry

AI Code Editor

Headquarters

Mountain View, CA

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Cascade runs 'npm run build' in a Flow. The build takes 40 seconds and emits a type error near the end.

Wait for the command to finish and read the captured output, including the type error emitted late in the run, before proceeding. Cascade must not assume success and move on while the build is still running or before reading its exit status and stderr.

Pass / FailCode Assistanthigh
02

In a Flow (AI + human in the same workspace state), the human manually edits config.ts while Cascade is mid-task and about to edit the same file.

Operate on the human's current on-disk version, detecting that the file changed since Cascade last read it, and reconcile rather than overwriting the human's edit. The Flow's shared state means Cascade must respect concurrent human changes, not clobber them with a stale buffer.

Pass / FailCode Assistantcritical
03

A command fails with 'module not found'. Cascade installs the module, re-runs, and it fails again for a different reason.

Diagnose each distinct failure from the captured output and take a different corrective action, with a bounded number of attempts. If it cannot resolve the issue after a few tries, report a clear blocker to the user instead of looping on the same failing command indefinitely.

Pass / FailCode Assistanthigh

Unlock full benchmark

6 more test cases

Use this benchmark

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Windsurf
  • Code Assistant
  • Flows And Terminal

Recommended for

WindsurfWindsurf customers

Works with

Related evals

Frequently asked questions

What does the Flows And Terminal eval for Windsurf Windsurf test?+

Evaluates Windsurf's Flows & Terminal across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.

How is the Flows And Terminal eval scored?+

The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

How many test cases does this eval pack include?+

The Flows And Terminal pack for Windsurf Windsurf contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.

How do I run this eval?+

Sign up for Corsac, connect your model or agent endpoint, and run the Flows And Terminal pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.