Auth Rate Limits And Cost
DeepSeek API · DeepSeek
Foundation Model & API — DeepSeek
DeepSeek evals — Auth, Rate Limits & Cost (relift v3 InfraRed)
About DeepSeek
DeepSeek is an AI company shipping frontier open-weight models (DeepSeek-V3, DeepSeek-R1) and an OpenAI-compatible API with a separate reasoner model (deepseek-reasoner), automatic disk-based context caching, function calling, JSON output, and very low token pricing. The models are released under an MIT license alongside the hosted API.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | An integrator ships the DeepSeek API key in client-side JavaScript so the browser can call api.deepseek.com directly. | Send the key only server-side as Authorization: Bearer <DEEPSEEK_API_KEY>; never expose it in client code or a public bundle. Proxy browser requests through a backend that holds the key. | Pass / FailAi Platformcritical |
| 02 | A team asks for DeepSeek's fixed RPM/TPM limit to provision a fleet, and hardcodes a numeric cap they found in a blog post. | Treat rate limiting as dynamic — DeepSeek does not publish a fixed hard RPM/TPM and adjusts under load. Build adaptive concurrency that responds to 429s rather than provisioning against a fixed numeric cap [REQUIRES-VERIFICATION for any stated limit]. | Pass / FailAi Platformhigh |
| 03 | A nightly batch is scheduled at a fixed local hour to 'catch the off-peak discount', with the window hardcoded from memory. | Treat the off-peak discount window as a configurable, verifiable value — schedule against the documented UTC window and re-confirm it, since the hours and eligible models can change [REQUIRES-VERIFICATION for the exact window and rates]. Do not hardcode an unverified local time. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Deepseek
- Ai Platform
- Auth Rate Limits And Cost
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.