Provider Routing And Fallbacks
Portkey AI Gateway · Portkey
AI Gateway — Portkey
Portkey evals — Provider Routing & Fallbacks (relift v3 InfraRed)
About Portkey
Portkey is an AI gateway for production LLM apps — a unified, OpenAI-compatible API across 200+ models with provider routing and fallbacks, semantic and simple caching, input/output guardrails (PII redaction, prompt-injection, content moderation), request-level observability and traces, a versioned prompt library, virtual keys with per-key budgets and rate limits, and workspace RBAC + audit logs.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Config strategy={mode:'fallback'} with targets=[anthropic_vk, openai_vk, bedrock_vk]; primary is Anthropic, fallbacks fire on listed status codes. | On primary upstream failure matching on_status_codes, the gateway tries the next target in declared order. Verify the cascade by reading x-portkey-provider on the response and the per-target attempt log. Order of targets[] is the contract — do not reorder between deploys without intent. | Pass / FailAi Platformcritical |
| 02 | Config strategy={mode:'loadbalance'} with targets weighted 70/20/10 across three virtual keys. | Weights are relative; gateway distributes traffic across targets in expectation matching the weights. Verify by aggregating x-portkey-provider over a representative sample. Do not expect per-request determinism — loadbalance is stochastic. | Pass / FailAi Platformhigh |
| 03 | Config strategy={mode:'conditional'} with query rules routing requests with metadata.env='prod' to a paid virtual key and metadata.env='dev' to a sandbox virtual key. | Send x-portkey-metadata JSON with env on every request. Conditional strategy evaluates query rules in declared order and routes to the matched target; a default target must be set for rule-miss. Verify with trace logs that prod metadata reaches the prod target. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Portkey
- Ai Platform
- Provider Routing And Fallbacks
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.