Evals for Atomic Chat

Platform-correct install path

Given a stated OS and architecture, direct the user to the matching desktop download, App Store / Google Play listing, or the corresponding shell/PowerShell install command.

Minimum requirement gating

Recognize when a device falls below documented floors (e.g. Intel Mac, pre-macOS 13) and say so rather than proceeding.

First-run path to a first message

Walk the documented three-step flow: download the app, pick a model, start chatting.

Account-free expectations

Explain that there is no registration, password, profile, or account to recover, and what that implies for setup and device migration.

Memory-aware recommendation

Map stated RAM/VRAM or Mac unified memory to a model size and quantization that fits, and decline to recommend builds that exceed it.

Model naming and tag literacy

Explain what the components of a local model name mean — family, parameter count, quantization/GGUF variant — so a user can compare builds.

Family selection by task

Differentiate the documented families by their stated strengths (e.g. reasoning/coding vs. multilingual vs. lightweight) without asserting benchmark rankings.

Context length and KV cache cost

Account for the fact that KV cache memory grows with context length when sizing a model against available memory.

Offline behavior boundaries

State clearly what requires connectivity (initial model and app download) versus what works with the internet off (inference on an already-downloaded model).

Generation settings

Explain the exposed controls — temperature, context size, quantization choice — and the practical tradeoff each one makes.

No-limits framing

Answer quota, cost and cap questions consistently: no token quotas or per-message caps, with hardware as the real constraint.

Local performance troubleshooting

Respond to slow generation, model-won't-load, or out-of-memory situations by pointing at size, quantization or context adjustments.

Agent launch setup

Guide a user from a running local model to a launched agent using the documented one-click path.

Local model as the agent backend

Explain that the agent is powered by a locally served Atomic Chat model rather than a hosted API, including the no-API-key implication.

Agent-to-model suitability

Advise which locally runnable model sizes are plausible backends for an agent given the user's hardware, without overpromising capability.

Scope of supported agents

Name the agents the product documents and avoid asserting support for ones it does not.

Telemetry precision

Separate chat/inference data (not transmitted) from anonymous usage analytics and website visit logs, without collapsing either into an absolute claim.

Open-source verifiability

Point to the public GitHub repository and the LICENSE file as the authoritative source for license terms rather than paraphrasing them as fact.

Acceptable use

Represent the terms' restrictions (e.g. illegal content) accurately alongside the product's uncensored-model positioning.

Local-vs-cloud comparison honesty

Make the documented local-vs-cloud contrasts without overstating them or claiming absolute privacy guarantees the policy explicitly disclaims.

Routing to the right guide

Point a user asking about a specific model family to the corresponding 'How to run X locally' guide instead of improvising steps.

Hardware requirement lookup

Surface documented disk and memory footprints for a named model build when the guides state them.

Quantization selection guidance

Explain how to choose among available GGUF/quantization builds for a target memory budget.

Concept explainers

Answer conceptual questions the guides cover — what a local LLM is, what a KV cache is and why it grows — at the depth a non-specialist needs.