Eval Library
P
For PortkeyAI Platform

Provider Routing And Fallbacks

Portkey AI Gateway · Portkey

AI Gateway — Portkey

Portkey evals — Provider Routing & Fallbacks (relift v3 InfraRed)

About Portkey

Portkey is an AI gateway for production LLM apps — a unified, OpenAI-compatible API across 200+ models with provider routing and fallbacks, semantic and simple caching, input/output guardrails (PII redaction, prompt-injection, content moderation), request-level observability and traces, a versioned prompt library, virtual keys with per-key budgets and rate limits, and workspace RBAC + audit logs.

Employees

~40

Industry

AI Gateway

Headquarters

San Francisco, CA

Website

portkey.ai

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Config strategy={mode:'fallback'} with targets=[anthropic_vk, openai_vk, bedrock_vk]; primary is Anthropic, fallbacks fire on listed status codes.

On primary upstream failure matching on_status_codes, the gateway tries the next target in declared order. Verify the cascade by reading x-portkey-provider on the response and the per-target attempt log. Order of targets[] is the contract — do not reorder between deploys without intent.

Pass / FailAi Platformcritical
02

Config strategy={mode:'loadbalance'} with targets weighted 70/20/10 across three virtual keys.

Weights are relative; gateway distributes traffic across targets in expectation matching the weights. Verify by aggregating x-portkey-provider over a representative sample. Do not expect per-request determinism — loadbalance is stochastic.

Pass / FailAi Platformhigh
03

Config strategy={mode:'conditional'} with query rules routing requests with metadata.env='prod' to a paid virtual key and metadata.env='dev' to a sandbox virtual key.

Send x-portkey-metadata JSON with env on every request. Conditional strategy evaluates query rules in declared order and routes to the matched target; a default target must be set for rule-miss. Verify with trace logs that prod metadata reaches the prod target.

Pass / FailAi Platformhigh

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Portkey
  • Ai Platform
  • Provider Routing And Fallbacks

Recommended for

Portkey AI GatewayPortkey customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.