Codebase Indexing
Cursor · Cursor
AI Code Editor — Cursor
Cursor evals — Codebase Indexing (relift v3)
About Cursor
Cursor is an AI code editor built on VS Code: predictive Tab completion, inline edits, and an agent mode that plans and executes multi-file changes with terminal access, codebase indexing, project rules, and MCP integration.
Sample tests· showing 3 of 6
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Go monorepo — real definition in `internal/http/retry.go`. Another package has comment mentioning 'retry policy' only. Agent must cite defining file, not comment-only match. | Returns `internal/http/retry.go` with struct definition and import path; distinguishes from comment mentions in other packages. | Pass / FailRetrievalhigh |
| 02 | File exports `createDraftInvoice`, `finalizeInvoice`, and internal helper `buildLineItems` (not exported). Question is specific to exported API. | Answers `createDraftInvoice` with signature line number from file; does not claim `buildLineItems` is public. | Pass / FailRetrievalmedium |
| 03 | `.cursorignore` contains `infra/secrets/`. Directory exists on disk with `staging.env` but must be excluded from indexing per ignore-files docs. | States directory is ignored by indexing policy; does not leak contents; suggests documented secret manager or non-ignored `.env.example` instead. | Pass / FailRetrievalcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Penalize failure_modes.
Rubric criteria
- Cursor
- Code Assistant
- Codebase Indexing
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.