Eval directory
Evals for Sourcegraph
8 evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Sourcegraph AI products.
About Sourcegraph
Sourcegraph is a code intelligence and AI coding platform: universal code search, precise code navigation, Cody chat grounded in your codebase, cross-repo batch changes, and the Amp autonomous agent — deployed across large enterprise codebases.
Available eval packs for Sourcegraph
8 packs ready to run.
Amp Autonomous Agent
Sourcegraph evals — Amp Autonomous Agent (relift v3 InfraRed)
Batch Changes
Sourcegraph evals — Batch Changes (relift v3 InfraRed)
Code Insights And Ownership
Sourcegraph evals — Code Insights & Ownership (relift v3 InfraRed)
Cody Autocomplete And Inline Edit
Sourcegraph evals — Cody Autocomplete & Inline Edit (relift v3 InfraRed)
Cody Chat And Context
Sourcegraph evals — Cody Chat & Context (relift v3 InfraRed)
Deployment Auth And Governance
Sourcegraph evals — Deployment, Auth & Governance (relift v3 InfraRed)
Precise Code Navigation
Sourcegraph evals — Precise Code Navigation (relift v3 InfraRed)
Universal Code Search
Sourcegraph evals — Universal Code Search (relift v3 InfraRed)
Why eval Sourcegraph AI
Sourcegraph's AI features ship behind brand promises about accuracy, safety, and reliability. Buyers and integrators need to know those promises hold up under adversarial prompts, edge-case workflows, and the long tail of real customer inputs — not just the demo path.
The Corsac eval library for Sourcegraph measures four dimensions teams care about most when deploying code assistant agents:
- Adversarial robustness — does the agent resist prompt injection, jailbreaks, and social-engineering attempts?
- Workflow quality— does it complete the task buyers were shown in the demo, on inputs that don't look like the demo?
- Safety gates — does it escalate or refuse when it should, and only then?
- Operator quality — does it preserve analyst trust by surfacing the right context at the right time?
Every eval pack above is hand-authored against Sourcegraph's public product surface and runnable in Corsac with your own data.