Evals for SambaNova

OpenAI-compatible endpoint migration

Swapping an existing OpenAI client over by setting the API key, base URL, and model id; what does and does not need to change.

Anthropic Messages API support

Recognizing that SambaCloud also exposes an Anthropic-compatible surface and when a caller would choose it.

Prompt caching behavior

Explaining prompt caching as a latency and cost lever on supported models without overstating guarantees.

Integrations and tooling

Getting started through named integrations such as CrewAI, Hugging Face, Cline, and AWS.

Supported model families

Answering which open-source families are available, including DeepSeek, Llama, Qwen, Gemma, MiniMax, and gpt-oss variants.

Modality coverage

Distinguishing text, image, and audio capability across catalog entries rather than assuming uniform support.

Model selection guidance

Matching a described workload (agentic, coding, high-volume chat) to a catalog model using grounded attributes.

Benchmark and speed claims

Attributing inference-speed claims to their stated third-party sources rather than presenting them as unsourced fact.

Dataflow versus kernel-by-kernel execution

Explaining PCU/PMU grids, the streaming pipeline, and why intermediate activations stay on-chip.

Data movement as the bottleneck

Articulating why off-chip data movement, not raw compute, is framed as the central cost.

Three-tier memory and model switching

How tiered memory supports very large models and fast switching between models.

SN50 versus SN40 generational deltas

Stating the specific claimed gains (compute, network bandwidth, scale to 256 accelerators) without inflating them.

Rack composition and scale-out

16 RDU chips per rack, scaling across racks and up to 256 accelerators over the interconnect.

Model and context ceilings

Handling stated limits on parameter count and context length for a single rack.

Power and cooling envelope

Air-cooled operation at roughly 10 kW per rack and its implications for existing data centers.

Deployment sizing walkthrough

Guiding a user through energy, floor space, and cluster scale estimates for a planned deployment.

Offering disambiguation

Correctly separating the developer API, the full-stack deployment, the managed in-customer-DC cloud, and the underlying rack.

On-prem versus hosted SambaStack

Representing both deployment options and the dedicated-infrastructure framing.

Turnkey timelines and operating model

Deployment-in-90-days and no-specialized-AI-expertise-required claims, stated as vendor claims.

Model bundles and hot-swapping

Pre-configured bundles per rack and swapping between bundles at inference time to serve more models per footprint.

Data privacy posture

Representing the stated position that SambaCloud does not see or collect customer data or prompts.

Sovereign and on-shore deployment

Keeping data, models, and compliance in-region for named government and enterprise geographies.

Attribution of third-party benchmarks

Citing Artificial Analysis and SemiAnalysis as the sources of speed claims rather than asserting them directly.

Corporate and customer claims

Handling financing, valuation, and named-customer statements as time-sensitive claims requiring a source.