Docker Hub And Registry
Docker (Engine, Hub, Build Cloud, Scout, Desktop, Model Runner) · Docker
Container Platform — Docker
Evaluates Docker's Docker Hub & Registry across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Container Platform eval coverage.
About Docker
Docker is the container platform — Docker Engine, Docker Desktop, Docker Hub registry, Docker Build Cloud for managed cloud builders, Docker Scout for image vulnerability scanning and supply-chain policy, Docker Compose for multi-container dev, and Docker Model Runner for local LLM inference. Millions of developers and tens of thousands of enterprises ship containerized software with Docker.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Production deployment references myorg/api:latest. A re-push of :latest overnight changes the resolved image and the next pod restart picks up an unintended version. | Production deploys pin by digest: myorg/api@sha256:<digest>. Resolve once at release time via 'docker buildx imagetools inspect' or 'docker manifest inspect' and store digest in the deployment manifest. Tags are mutable references; digests are immutable content addresses. | Pass / FailAi Platformcritical |
| 02 | CI cluster pulls public Docker Hub images on every job. Sporadic builds fail with 'toomanyrequests: You have reached your pull rate limit'. | Authenticate pulls with a Hub account or org so quota is per-user not per-IP. For sustained CI, mirror upstream images into a private registry (Hub org, registry pull-through cache, or self-hosted) and pin manifests by digest. Numeric limits [REQUIRES-VERIFICATION] against docs.docker.com/docker-hu… | Pass / FailAi Platformhigh |
| 03 | CI uses 'docker login -u myuser -p $DOCKERHUB_PASSWORD'. Security review flags use of the personal password. | Create a Personal Access Token in Docker Hub with scoped permissions (read-only for pulls, read-write for pushes). Use 'docker login -u myuser -p $DOCKERHUB_PAT'. Tokens are revocable per-CI without rotating the account password. Org-managed deploys should use Organization Access Tokens with role s… | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Docker
- Ai Platform
- Docker Hub And Registry
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Docker Hub And Registry eval for Docker Docker (Engine, Hub, Build Cloud, Scout, Desktop, Model Runner) test?+
Evaluates Docker's Docker Hub & Registry across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Container Platform eval coverage.
How is the Docker Hub And Registry eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Docker Hub And Registry pack for Docker Docker (Engine, Hub, Build Cloud, Scout, Desktop, Model Runner) contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Docker Hub And Registry pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.