Eval directory

Evals for Hyperbolic

Eval coverage for Hyperbolic, mapped from its public product surface.

About Hyperbolic

Hyperbolic is an open-access AI cloud that provides on-demand GPU compute and model inference to AI builders. It offers three consumption models — on-demand GPU instances, reserved clusters, and private cloud via a supplier network — covering H100, H200, and B200 hardware. It also provides dedicated, single-tenant model hosting for production inference and an OpenAI-compatible serving API.

Industry

GPU and AI cloud infrastructure

Website

www.hyperbolic.ai

Use the eval library for Hyperbolic

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Hyperbolic?

6 scoring areas · 22 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

On-Demand GPU Provisioning

Self-serve launch and lifecycle of H100, H200, and B200 instances in minutes with no commitment, managed from the dashboard and billed by usage.

“Hyperbolic gives 250,000+ builders affordable on-demand GPUs to train fast, serve via an OpenAI-compatible API” www.hyperbolic.ai

Mapped capabilities

4 capabilities

Supported GPU tiers and availability
Which hardware (H100, H200, B200) can be provisioned on demand and how capacity is sourced through the provider network.
Launch and teardown flow
Creating an instance without sales calls, forms, or quota negotiation; deploying state and time-to-ready expectations.
Scale up and down
Adjusting usage as workloads change and paying only for compute consumed.
Fit-for-workload guidance
When on-demand is the right choice — experimentation, training, fine-tuning, inference, short-term production.

Illustrative example

Input: I need GPUs today for a fine-tuning run but I can't sign a contract. What hardware can I launch on demand, and what do I have to commit to?
Expected behavior: Names H100, H200, and B200 as on-demand options launchable in minutes, states that no commitment is required and billing is usage-based, and points to reserved clusters or private cloud only as alternatives for longer-term needs.

Reserved and Private Capacity

The two commitment-based consumption models: reserved clusters at discounted rates under one-year terms, and private cloud through the supplier network for the longest commitments.

“Reserve dedicated capacity at discounted rates with commitments under one year.” www.hyperbolic.ai

Mapped capabilities

4 capabilities

Reserved cluster terms
Discounted dedicated capacity with sub-one-year commitments for steady production workloads.
Private cloud via supplier network
Dedicated long-term infrastructure at the lowest cost for teams needing predictable capacity.
Choosing among the three models
Matching on-demand, reserved, or private cloud to workload steadiness, budget, and commitment appetite.
Predictability of access
How reservation converts variable availability into guaranteed GPU access.

OpenAI-Compatible Serving API

The inference API surface builders use to serve models, presented as OpenAI-compatible so existing client code can target it.

Mapped capabilities

3 capabilities

API compatibility contract
What OpenAI-compatible means for request and response shape and existing SDK reuse.
Migration from an existing OpenAI client
Repointing an application to the Hyperbolic serving endpoint.
Serving at production scale
Using the API to move from experimentation to served production traffic.

Illustrative example

Input: My app already uses the OpenAI Python SDK. How much of my code changes if I serve the model on Hyperbolic instead?
Expected behavior: States that the serving API is OpenAI-compatible, so the existing SDK client is reused with the base URL and API key repointed rather than rewritten. Does not fabricate endpoint paths, model identifiers, or parameters absent from the provided context.

Dedicated Model Hosting

Single-tenant model hosting for production inference: reserved GPU capacity and isolation instead of shared endpoints, with predictable spend and no self-managed hardware.

“Get access to Hyperbolic's supplier network for dedicated, long-term infrastructure at the lowest cost.” www.hyperbolic.ai

Mapped capabilities

4 capabilities

Single-tenant isolation
Reduced security surface versus shared environments for sensitive prompts or regulated data.
Latency and throughput stability
Avoiding tail-latency jitter and noisy-neighbor throughput drops from multi-tenant contention.
Cost predictability at steady scale
Reserved capacity versus usage-based billing for steady high-volume inference and unit-economics modeling.
Shared-endpoint tradeoffs
When shared inference is acceptable and when its limitations disqualify it.

Accounts, Organizations, and Billing

Team-level account structure and the usage-based spend model that governs how builders access and pay for compute.

“Scale usage up or down as needed and only pay for the compute you use.” www.hyperbolic.ai

Mapped capabilities

3 capabilities

Organizations
Team-level account grouping introduced for shared access to compute.
Usage-based billing
Paying only for compute used on on-demand instances versus committed pricing.
Self-serve onboarding
Getting to first instance without procurement cycles or sales contact.

Hardware Selection and Monitoring Guidance

The advisory surface Hyperbolic publishes to help buyers choose architectures and diagnose utilization — pricing explainers, Hopper vs Blackwell comparisons, memory concepts, and GPU monitoring.

“Launch H100, H200, B200, and other high-performance GPUs in minutes with no commitment.” www.hyperbolic.ai

Mapped capabilities

4 capabilities

H200 economics: buy vs rent
Purchase price ranges by form factor versus hourly rental in a supply-constrained market.
H100 vs H200 vs Blackwell
Memory capacity and bandwidth differences and when the H200 premium is justified.
Dedicated vs shared GPU memory
VRAM versus system-RAM spillover and its effect on training and inference performance.
Utilization diagnostics
SM efficiency, memory bandwidth, and bottleneck identification for underutilized GPUs.

Coverage is mapped from Hyperbolic's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Hyperbolic test?+

The coverage map is generated from Hyperbolic's own public product surface (GPU and AI cloud infrastructure): 6 scoring areas — On-Demand GPU Provisioning, Reserved and Private Capacity, and OpenAI-Compatible Serving API, and more — spanning 22 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Hyperbolic evals scored?+

Every case generated for Hyperbolic — across On-Demand GPU Provisioning and Reserved and Private Capacity and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Hyperbolic library include?+

The full Hyperbolic library is built on request. The coverage map spans 6 areas and 22 capabilities (for example, Supported GPU tiers and availability and Launch and teardown flow under On-Demand GPU Provisioning); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Hyperbolic or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Hyperbolic areas and set them up in a Corsac workspace, where you can run every test case against Hyperbolic or your own agent with your own data.