Usage Accounting Cost Attribution
OpenRouter · OpenRouter
OpenRouter evals — Usage Accounting & Cost Attribution (relift v3)
About OpenRouter
OpenRouter is a unified LLM routing layer that gives developers access to hundreds of models through a single OpenAI-compatible API. It automatically routes requests to the best available provider, with fallback handling and transparent per-token pricing.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Finance reconciles nightly spend vs internal token counters; is_byok false on pooled routes. | usage.cost reflects OpenRouter-billed amount for platform-routed traffic; reconcile with dashboard credits and generation API records. | Pass / FailBillingcritical |
| 02 | Prompt cache hits spike after RAG template stabilization; need accurate marginal cost. | Explain cached_tokens reduces billed prompt cost per documented pricing; do not sum cached_tokens into regular prompt_tokens twice. | Pass / FailBillinghigh |
| 03 | Dispute on invoice line item; only generation id logged in proxy. | Pull model slug, provider path, token counts, cost, timestamps, and BYOK flags matching completion response usage. | Pass / FailObservabilitymedium |
Rubric criteria
- Openrouter
- Ai Platform
- Usage Accounting Cost Attribution
Recommended for
Works with
Related evals
Accounting Close Controls
Operational response/safety eval for Puzzle covering accounting close controls.
View Accounting & FinanceAudit Readiness Traceability
Wave 2 production eval for Puzzle focused on audit readiness traceability.
View Accounting & FinanceExpert Safety Gate
High-confidence expert safety gate eval for Puzzle.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.