Universal Code Search
Sourcegraph (Cody + Amp) · Sourcegraph
Code Intelligence — Sourcegraph
Sourcegraph evals — Universal Code Search (relift v3 InfraRed)
About Sourcegraph
Sourcegraph is a code intelligence and AI coding platform: universal code search, precise code navigation, Cody chat grounded in your codebase, cross-repo batch changes, and the Amp autonomous agent — deployed across large enterprise codebases.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator runs `log15.Error( lang:go` across a Sourcegraph instance synced with 600 mirror forks of acme/platform. Result set is 40k+ hits dominated by forks; the actual canonical repo is buried. | Rewrite as `repo:^github\.com/acme/platform$ lang:go log15.Error(` — anchor the repo: regex with ^...$ to exclude forks. Keyword patterntype is the default since v5.3 per docs/code-search/queries; no patterntype filter is required for the literal substring `log15.Error(`. | Pass / FailCode Assistanthigh |
| 02 | Compliance asks: 'when was the AWS access key prefix `AKIA` first introduced to repo acme/infra and where?' Operator only knows code search, not git log archaeology. | Use `type:diff repo:^github\.com/acme/infra$ AKIA` (optionally with `after:` / `before:` time filters) to surface the commits whose diffs added or removed `AKIA`. Cite docs/code-search/queries diff-and-commit search. Pivot to `type:commit` only if the secret was added via commit message, not file c… | Pass / FailCode Assistantcritical |
| 03 | Operator writes `foo AND bar OR baz` expecting `(foo AND bar) OR baz`, but results suggest a different grouping. | Per docs/code-search/queries boolean operators, AND binds tighter than OR, so the query already parses as `(foo AND bar) OR baz`. To make precedence explicit and survive future query-parser changes, wrap with parentheses: `(foo AND bar) OR baz`. Use uppercase operators. | Pass / FailCode Assistantmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Sourcegraph
- Code Assistant
- Universal Code Search
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.