Cicd Auth And Governance
Antithesis · Antithesis
Autonomous Deterministic Testing — Antithesis
Antithesis evals — CI/CD, Auth & Governance (relift v3 InfraRed)
About Antithesis
Antithesis is an autonomous, deterministic software testing platform. It runs a system-under-test (delivered as containers) inside a deterministic hypervisor that controls every source of nondeterminism — clock, network, thread scheduling, and randomness — then autonomously explores the state space to find bugs, including hard concurrency and distributed-systems failures. Every bug it finds is perfectly reproducible from a recorded seed, with time-travel debugging across a branching 'multiverse' of explored futures, plus fault injection (network partitions, crashes, disk faults, clock skew) and SDK assertions (Always / Sometimes / Reachable). [REQUIRES-VERIFICATION] employees, funding, and exact HQ location were not network-verifiable at authoring time.
Employees
[REQUIRES-VERIFICATION]
Industry
Software Testing / Developer Tools
Headquarters
United States [REQUIRES-VERIFICATION]
Website
antithesis.comSample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator wants every release candidate to run an Antithesis test before promotion to production. | Wire an Antithesis run into the CI/CD pipeline as a gate: build the pinned SUT image, trigger a run with a defined budget, and block promotion on any property violation or on a Sometimes guard never firing (coverage regression). [REQUIRES-VERIFICATION] for the exact CI trigger/integration mechanism… | Pass / FailAi Platformhigh |
| 02 | A full deep exploration takes too long for per-PR CI but the team still wants deep coverage. | Split the workload: a short, bounded per-PR run that catches regressions fast, plus a longer scheduled (nightly/weekly) soak that explores deeper. Pin failing seeds from the soak back into the fast PR suite as regression tests. Size each tier to its purpose rather than forcing one budget to serve b… | Pass / FailAi Platformmedium |
| 03 | CI needs to authenticate to the Antithesis platform. A developer pastes a long-lived token into the pipeline YAML in plaintext. | Store the platform credential in the CI secret store (not in committed YAML), scope it to the minimum needed (trigger runs / read reports for this project), and rotate it on a schedule. [REQUIRES-VERIFICATION] for the exact auth/token model and available scopes. Never echo the token into build logs. | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Antithesis
- Ai Platform
- Cicd Auth And Governance
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.