Eval Library
S
For SourcegraphCode Assistant

Code Insights And Ownership

Sourcegraph (Cody + Amp) · Sourcegraph

Code Intelligence — Sourcegraph

Sourcegraph evals — Code Insights & Ownership (relift v3 InfraRed)

About Sourcegraph

Sourcegraph is a code intelligence and AI coding platform: universal code search, precise code navigation, Cody chat grounded in your codebase, cross-repo batch changes, and the Amp autonomous agent — deployed across large enterprise codebases.

Employees

~150

Industry

Code Intelligence

Headquarters

San Francisco, CA

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Operator wants a weekly trend of `TODO(security)` count across the codebase. Defines a Code Insight with a search query and a 30-day window.

Per docs/code_insights/references, a search-based insight runs the query at each sample point against historical commits, returning a time series. Verify the query is well-scoped (`type:diff` for added/removed counts or plain content for current state). Sampling cadence and backfill horizon are con…

Pass / FailCode Assistantmedium
02

A directory has no CODEOWNERS entry but the site admin has assigned ownership via the Own UI to team-platform. Operator expects the assignment to surface.

Per docs/own, ownership combines CODEOWNERS rules with assigned ownership; when CODEOWNERS is silent, the assigned owner is the authoritative source. Verify in the Own UI that both the rule and the assignment surface, and that the dashboard cites the assignment provenance.

Pass / FailCode Assistanthigh
03

Operator wants a chart of language version counts: each capture of `node>=([0-9.]+)` in `package.json` is grouped into a series.

Per docs/code_insights, capture-group insights use regexp patterntype with a single capture group whose match becomes the series key. Verify the regexp is anchored and the capture group is non-greedy enough to avoid swallowing trailing JSON. Cite the capture-group insight guide.

Pass / FailCode Assistantmedium

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Sourcegraph
  • Code Assistant
  • Code Insights And Ownership

Recommended for

Sourcegraph (Cody + Amp)Sourcegraph customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.