Eval directory

Evals for DigitalOcean

Eval coverage for DigitalOcean, mapped from its public product surface.

About DigitalOcean

DigitalOcean markets itself as an "AI-Native Cloud": a single integrated platform spanning five layers, from owned data center silicon and core cloud primitives up through an inference engine, data/learning services, and managed agents. Core cloud products include Droplets (virtual machines), App Platform, Functions (serverless), managed databases, and S3-compatible Spaces object storage, alongside GPU Droplets and serverless inference for AI/ML workloads. Pricing is positioned as simple and predictable, with flat monthly rates, monthly caps, and pay-per-use token billing for inference and evaluations.

Industry

AI-native cloud infrastructure platform (GPUs, inference, and managed cloud compute/storage)

Website

www.digitalocean.com

Use the eval library for DigitalOcean

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for DigitalOcean?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Core Cloud Compute

The compute primitives DigitalOcean sells as its "Core Cloud" layer: Droplets, App Platform, Kubernetes, and serverless Functions, including their positioning by workload type and their reliability commitments.

Mapped capabilities

4 capabilities

Droplet types and workload fit
Basic, General Purpose, CPU-Optimized, Memory-Optimized, Storage-Optimized, and the use cases each is positioned for; Premium variants.
Droplet reliability and included resources
99.99% uptime SLA, 500 GiB/month free outbound transfer, included monitoring and firewalls.
Private Droplets
Generally available compute instances architecturally isolated from the public internet, with no public IP and no price difference.
Functions and serverless execution model
On-demand invocation in response to web events, automatic scaling without pre-provisioned resources, deployment as App Platform components.

Inference Engine and Model Routing

The serverless and dedicated inference layer: a single endpoint fronting a large model catalog, with the Inference Router selecting a model per request.

“Serverless inference, intelligent routing, and 80+ models. No infrastructure to wrangle.” www.digitalocean.com

Mapped capabilities

4 capabilities

Model catalog breadth
Open-weighted and frontier models on one endpoint; catalog size as stated on the marketing and products pages, including newly listed models.
Inference deployment modes
Serverless, dedicated, and batch inference options and how they differ.
Inference Router behavior
Intelligent routing of each request to an appropriate model, framed as paying only for the intelligence used.
Single-API deployment claims
Deploy with one API call, no infrastructure management, no egress fees between platform layers.

Evaluations

The managed LLM-as-a-Judge evaluation product for comparing model quality and performance before production, including what it can be pointed at and how it is billed.

“Run LLM-as-a-Judge evaluations on any LLM, dedicated endpoint, or inference routing policy on DigitalOcean Inference” www.digitalocean.com

Mapped capabilities

4 capabilities

Evaluation targets
Running evaluations against any LLM, a dedicated endpoint, or an inference routing policy on DigitalOcean Inference.
Quality and performance comparison
Comparing quality and performance across candidates in a single run.
Custom datasets and configuration
Bringing your own datasets and evaluation configurations.
Evaluation billing basis
Charges driven by candidate plus judge token usage rather than a flat rate.

Storage and Managed Data Services

S3-compatible Spaces object storage with its built-in CDN and cold tier, plus the managed database engines offered alongside it.

“Spaces offer cold storage for $0.007/GiB per month.” www.digitalocean.com

Mapped capabilities

4 capabilities

Spaces plan contents and overage
$5/month base bucket with 250 GiB storage and 1 TiB outbound transfer; per-GiB overage; free inbound; up to 100 buckets.
S3 compatibility and ecosystem
Fully S3-compatible API usable with existing S3 tools, libraries, plugins, and extensions.
Cold storage tier
$0.007/GiB per month, one free full-bucket retrieval per month, 30-day minimum retention, instant retrieval for petabyte-scale infrequently accessed data.
Managed database engines
MongoDB, MySQL, PostgreSQL, Valkey, Kafka, and OpenSearch as listed managed offerings.

Illustrative example

Input: What do I get for the $5/month Spaces Object Storage plan, and what happens if I store more than that?
Expected behavior: States the $5/month plan includes 250 GiB of storage and 1 TiB of outbound transfer, and that additional storage is $0.02/GiB. Notes inbound bandwidth to Spaces is free. Does not invent an outbound overage rate not present in the source.

Pricing, Billing, and Cost Estimation

The "simple, predictable pricing" surface: entry prices per product, billing mechanics and their effective dates, and the self-serve pricing calculator.

“Get dependable uptime with our 99.99% SLA, simple security tools, and predictable monthly pricing” www.digitalocean.com

Mapped capabilities

4 capabilities

Entry price points by product
Droplets from $4/month, App Platform from $0/month, AI/ML inference from $0.05/M tokens, Spaces from $5/month.
GPU Droplet pricing structure
From $1.91/GPU/hour on multi-month commitment versus on-demand from $0.76/GPU/hour, and the conditions attached to each.
Droplet billing granularity
Per-second billing effective January 1, 2026, with a minimum charge of 60 seconds or $0.01, whichever is higher.
Pricing calculator scope
Which product families the estimator covers — compute, databases, networking, storage, developer tools, support plans — and how estimate line items are added or removed.

Illustrative example

Input: I run a batch job on a Droplet that finishes in 20 seconds. How much Droplet time am I billed for?
Expected behavior: Explains that as of January 1, 2026 Droplets bill per second with a minimum charge of 60 seconds or $0.01, whichever is higher, so a 20-second run is billed at that minimum rather than 20 seconds of usage.

Managed Agents and Developer Workflow

The top platform layer plus the day-to-day interfaces developers use: agent runtimes and starter kits, and the management surfaces for provisioning and scaling.

Mapped capabilities

4 capabilities

Launchpad starter kits
Pre-configured starting points for RAG assistants, data pipelines, and observability aimed at users without infrastructure expertise.
Agent runtime components
The named Managed Agents products listed on the platform page, including Open Harness, Sandbox, Toolbox, and State.
Management interfaces
UI, CLI, API, and Terraform Provider as supported ways to deploy and manage Droplets.
Integrated app and data workflow
Adding a managed database during app creation and securing app-to-database connections with features such as trusted sources.

Coverage is mapped from DigitalOcean's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for DigitalOcean test?+

The coverage map is generated from DigitalOcean's own public product surface (AI-native cloud infrastructure platform (GPUs, inference, and managed cloud compute/storage)): 6 scoring areas — Core Cloud Compute, Inference Engine and Model Routing, and Evaluations, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the DigitalOcean evals scored?+

Every case generated for DigitalOcean — across Core Cloud Compute and Inference Engine and Model Routing and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the DigitalOcean library include?+

The full DigitalOcean library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Droplet types and workload fit and Droplet reliability and included resources under Core Cloud Compute); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against DigitalOcean or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped DigitalOcean areas and set them up in a Corsac workspace, where you can run every test case against DigitalOcean or your own agent with your own data.