Eval directory · Code Assistant
Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Cursor AI products.
Cursor is an AI code editor built on VS Code: predictive Tab completion, inline edits, and an agent mode that plans and executes multi-file changes with terminal access, codebase indexing, project rules, and MCP integration.
Use the eval library for Cursor
All 51 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.
Related in Code Assistant
All evals →Coverage map
9 areas · 51 graded scenarios
Every eval set is graded on
Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls
01
Evaluates Cursor's Codebase Indexing across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
6 scenarios
02
Evaluates Cursor's Composer & Agent Mode across 7 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
7 scenarios
03
Evaluates Cursor's Inline Edit (Cmd-K) across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
6 scenarios
04
Evaluates Cursor's MCP Integration across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
6 scenarios
05
Evaluates Cursor's Model Selection & Routing across 5 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
5 scenarios
06
Evaluates Cursor's Privacy & Edit Safety across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
6 scenarios
07
Evaluates Cursor's Project Rules across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
6 scenarios
08
Evaluates Cursor's Tab Completion across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Code Editor eval coverage.
Mapped capabilities
6 scenarios
09
Smoke test for AI coding / IDE assistant completion correctness.
Mapped capabilities
3 scenarios
Example criterion: Cursor generates correct, maintainable code completions that satisfy task intent without unsafe patterns.
Each eval pack tests Cursor's public product surface — including Codebase Indexing, Composer Agent, Inline Edit — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.
Every eval set is graded the same way: pass/fail checks plus an LLM judge scoring 1–5 against each case's expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.
The Cursor eval library includes 51 graded test cases across 9 eval packs. Each case defines an input, expected behavior, and pass/fail criteria.
Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run every test case against Cursor or your own agent with your own data.
