Eval directory · Security Operations
Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Chainguard AI products.
Chainguard is a software supply chain security company that provides hardened, minimal container images with verifiable provenance. Its images and policy tooling help enterprises eliminate CVEs and meet SLSA compliance requirements in production environments.
Use the eval library for Chainguard
All 73 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.
Related in Security Operations
All evals →Coverage map
6 areas · 73 graded scenarios
Every eval set is graded on
Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls
01
Evaluates Chainguard's Advisory & Image Operations across 7 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Security infrastructure eval coverage.
Mapped capabilities
7 scenarios
Public sample case
02
Evaluates Chainguard's Custom Assembly & Builds across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Security infrastructure eval coverage.
Mapped capabilities
6 scenarios
Public sample case
03
Evaluates Chainguard's Policy Gates & Image Lifecycle across 15 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Security infrastructure eval coverage.
Mapped capabilities
15 scenarios
Public sample case
04
Evaluates Chainguard's Registry Auth & Pull Identity across 15 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Security infrastructure eval coverage.
Mapped capabilities
15 scenarios
05
Evaluates Chainguard's SBOM & Provenance Artifacts across 14 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Security infrastructure eval coverage.
Mapped capabilities
14 scenarios
06
Evaluates Chainguard's Signature & Attestation Verification across 16 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Security infrastructure eval coverage.
Mapped capabilities
16 scenarios
Each eval pack tests Chainguard's public product surface — including Advisory Image Operations, Custom Assembly Builds, and Policy Gates Image Lifecycle — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.
Pass/fail checks plus an LLM judge scoring 1–5 against each of the 73 Chainguard cases — from Signature Attestation Verification (16 scenarios) down to the smallest pack — its own expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published to the Chainguard library.
The Chainguard eval library includes 73 graded test cases across 6 eval packs, the largest being Signature Attestation Verification with 16 scenarios. Each case defines an input, expected behavior, and pass/fail criteria.
Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run all 6 Chainguard packs — Advisory Image Operations and Custom Assembly Builds and the rest — against Chainguard or your own agent with your own data.
