All evals
Defense Unicorns

Eval directory

Evals for Defense Unicorns

Eval coverage for Defense Unicorns, mapped from its public product surface.

About Defense Unicorns

UDS is Defense Unicorns' software delivery platform for military and defense systems, covering build, package, deploy, and manage workflows across cloud, edge, and airgapped environments. It bundles hardened container images, SBOM/CVE scanning, and ATO compliance artifacts, and is offered as a free AGPLv3 base plus paid UDS Fleet (distributed edge fleets) and UDS Enterprise tiers. The company positions the platform as portable, open source, and operable fully offline in DDIL conditions.

Industry

airgap software delivery and compliance platform for defense

Use the eval library for Defense Unicorns

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

Generate your own →

Coverage map

What would you measure for Defense Unicorns?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

  • Adversarial robustness
  • Workflow quality
  • Safety gates
  • Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

01

Build, Package, Deploy Workflow

The core four-stage software workflow the platform advertises: making compliance and security the build baseline, bundling once for any environment, and delivering to target systems via UDS Core and the UDS CLI.

We make running modern software on military systems easy. defenseunicorns.com

Mapped capabilities

4 capabilities

  • Bundle-once portability

    Explaining or guiding a single bundle targeting multiple environments, from secure cloud to tactical edge, without per-environment rebuilds.

  • UDS CLI task guidance

    Correct, tier-agnostic guidance for CLI-driven package and deploy steps, which the pricing page lists in every tier including free Base.

  • Deploy-time preconditions

    Surfacing what must be present before a deploy (bundle, target cluster, registry availability) rather than assuming a connected default.

  • Environment targeting

    Distinguishing cloud, on-prem, and edge targets and adjusting instructions to the stated destination.

02

Airgap and DDIL Operation

The company's central differentiator: transporting cloud-native software into airgapped environments and operating fully offline in denied, degraded, intermittent, and limited-bandwidth conditions with no Internet required.

Operates fully offline in DDIL environments, no Internet required defenseunicorns.com

Mapped capabilities

4 capabilities

  • Offline-first defaults

    Never prescribing steps that silently require Internet access, external registries, or license phone-home when the user states an airgapped context.

  • Media transfer across the gap

    Handling the move of artifacts from a connected side to a disconnected side as an explicit, described step.

  • Degraded-link behavior

    Reasonable guidance when connectivity is intermittent or bandwidth-limited rather than fully absent.

  • Disconnected failure recovery

    Recovering a partial or failed deploy without assuming the operator can pull anything new from outside the enclave.

Illustrative example

Input
Our enclave has no Internet at all. Walk me through deploying a UDS bundle we already built to a disconnected edge cluster.
Expected behavior
Gives an offline path: move the existing bundle across the airgap on approved media, then deploy from local artifacts. Every step works without Internet, and any external dependency is called out as needing to be staged beforehand on the connected side.

03

Security and Compliance Artifacts

SBOM and CVE scanning available in all tiers, plus paid-tier Unicorn-hardened and FIPS-validated images, ATO compliance artifacts (SCTM, SBOM, system architecture diagrams), and the stated NIST SP 800-53 out-of-the-box control posture.

UDS makes approvals, vulnerability scanning, and compliance simple. defenseunicorns.com

Mapped capabilities

4 capabilities

  • SBOM and CVE scan output

    Explaining what the scans cover and where results fit in the workflow, without overstating findings as an authorization decision.

  • ATO artifact package

    Correctly naming the artifacts described (SCTM, SBOM, system architecture diagrams) and their role in accelerating ATO timelines.

  • Hardened and FIPS image claims

    Stating hardened near-zero-CVE and FIPS-validated images as paid-tier features for the core runtime, and not as free Base capabilities.

  • Control-framework framing

    Referencing NIST SP 800-53 alignment as a starting baseline rather than asserting a granted ATO.

04

Licensing Tiers and Entitlements

The three-tier model — free AGPLv3 Base, commercial UDS Fleet for distributed edge fleets, and commercial UDS Enterprise for enterprise-scale cloud and on-prem — each with a specific published feature matrix.

Package once, deploy anywhere — from secure cloud to tactical edge defenseunicorns.com

Mapped capabilities

4 capabilities

  • Feature-to-tier mapping

    Accurately placing each listed feature in Base, Fleet, and/or Enterprise per the published comparison table.

  • Fleet-only capabilities

    Correctly isolating features the table marks for Fleet alone, such as UDS Connect (Android), UDS System Image, and Resilient Data Mesh.

  • Enterprise-only capabilities

    Correctly isolating Enterprise-only entries such as UDS Registry, Compliance Assessment Tool, and Infrastructure-as-Code modules.

  • Open-source obligations

    Representing the AGPLv3 nature of the free Base tier without inventing permissions or restrictions.

Illustrative example

Input
We're on the free AGPLv3 base. Can our operators use UDS Connect on Android tablets in the field, or do we need to buy something?
Expected behavior
Answers no: the published matrix puts UDS Connect (Android) in Fleet only. Notes the web version of Connect is in Fleet and Enterprise, and that Base covers UDS Core, UDS CLI, and SBOM/CVE scans. Points to the sales contact for Fleet.

05

Fleet Management and Observability

Paid-tier operational surfaces: UDS Command as a centralized interface for fleet-wide observability and management of distributed systems, and UDS Connect for one-to-one browser-based edge system management on tablet and desktop.

Mapped capabilities

4 capabilities

  • Centralized fleet visibility

    Describing UDS Command's fleet-wide observability and management scope over distributed UDS systems.

  • One-to-one edge access

    Describing UDS Connect as browser-based single-system management for tablet and desktop, with Android as a Fleet-tier extension.

  • Real-time threat monitoring

    Framing the Manage stage as ongoing observability to stay ahead of threats, without promising specific detections.

  • Support escalation path

    Routing incidents to the stated General Support offering, including its 24/7 response commitment for critical issues.

06

Commercial Terms and Sales Handoff

The published License and Service Agreement (effective March 1, 2026), Order Form mechanics, and the guided contact paths for Fleet and Enterprise conversations.

Mapped capabilities

4 capabilities

  • Order Form mechanics

    Explaining that fees follow quantities purchased rather than actual usage, per the published terms.

  • Cancellation and refund posture

    Accurately stating that orders are non-cancelable, fees are non-refundable, and quantities cannot be decreased mid-term.

  • Agreement versioning

    Citing the correct effective-date version and acknowledging that a prior version exists.

  • Qualified handoff

    Directing pricing-specific and contract-specific questions to the sales contact path instead of quoting invented figures.

Coverage is mapped from Defense Unicorns's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Defense Unicorns test?+

The coverage map is generated from Defense Unicorns's own public product surface (airgap software delivery and compliance platform for defense): 6 scoring areas — Build, Package, Deploy Workflow, Airgap and DDIL Operation, and Security and Compliance Artifacts, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Defense Unicorns evals scored?+

Every case generated for Defense Unicorns — across Build, Package, Deploy Workflow and Airgap and DDIL Operation and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Defense Unicorns library include?+

The full Defense Unicorns library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Bundle-once portability and UDS CLI task guidance under Build, Package, Deploy Workflow); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Defense Unicorns or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Defense Unicorns areas and set them up in a Corsac workspace, where you can run every test case against Defense Unicorns or your own agent with your own data.