Code Insights And Ownership
Sourcegraph (Cody + Amp) · Sourcegraph
Code Intelligence — Sourcegraph
Sourcegraph evals — Code Insights & Ownership (relift v3 InfraRed)
About Sourcegraph
Sourcegraph is a code intelligence and AI coding platform: universal code search, precise code navigation, Cody chat grounded in your codebase, cross-repo batch changes, and the Amp autonomous agent — deployed across large enterprise codebases.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator wants a weekly trend of `TODO(security)` count across the codebase. Defines a Code Insight with a search query and a 30-day window. | Per docs/code_insights/references, a search-based insight runs the query at each sample point against historical commits, returning a time series. Verify the query is well-scoped (`type:diff` for added/removed counts or plain content for current state). Sampling cadence and backfill horizon are con… | Pass / FailCode Assistantmedium |
| 02 | A directory has no CODEOWNERS entry but the site admin has assigned ownership via the Own UI to team-platform. Operator expects the assignment to surface. | Per docs/own, ownership combines CODEOWNERS rules with assigned ownership; when CODEOWNERS is silent, the assigned owner is the authoritative source. Verify in the Own UI that both the rule and the assignment surface, and that the dashboard cites the assignment provenance. | Pass / FailCode Assistanthigh |
| 03 | Operator wants a chart of language version counts: each capture of `node>=([0-9.]+)` in `package.json` is grouped into a series. | Per docs/code_insights, capture-group insights use regexp patterntype with a single capture group whose match becomes the series key. Verify the regexp is anchored and the capture group is non-greedy enough to avoid swallowing trailing JSON. Cite the capture-group insight guide. | Pass / FailCode Assistantmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Sourcegraph
- Code Assistant
- Code Insights And Ownership
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.