Unified Api And Gateway Proxy
Portkey AI Gateway · Portkey
AI Gateway — Portkey
Portkey evals — Unified API & Gateway Proxy (relift v3 InfraRed)
About Portkey
Portkey is an AI gateway for production LLM apps — a unified, OpenAI-compatible API across 200+ models with provider routing and fallbacks, semantic and simple caching, input/output guardrails (PII redaction, prompt-injection, content moderation), request-level observability and traces, a versioned prompt library, virtual keys with per-key budgets and rate limits, and workspace RBAC + audit logs.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Team migrates an existing OpenAI-SDK app to Portkey by changing only base_url to https://api.portkey.ai/v1 and passing the Portkey key as the OpenAI bearer, but forgets x-portkey-virtual-key. | Set base_url=https://api.portkey.ai/v1 and Authorization: Bearer <portkey-key>, then pass x-portkey-virtual-key (or x-portkey-config) on every request so the gateway knows which upstream provider credentials to use. Without a virtual key or config, the gateway returns an error rather than silently … | Pass / FailAi Platformhigh |
| 02 | Native Portkey SDK call omits the x-portkey-api-key header and relies on an environment variable from a prior process; the value is unset in the deployed container. | Construct the Portkey client with an explicit api_key argument (or fail fast at startup when the env var is missing). The gateway rejects requests without x-portkey-api-key with a clear auth error — surface that error to the caller; do not retry with empty auth. | Pass / FailAi Platformcritical |
| 03 | Operator needs Anthropic-only fields (top_k, anthropic-beta header for prompt caching) on a Portkey-proxied request. | Per Portkey docs, provider-specific parameters can be passed in the body and provider-specific headers via the request headers; Portkey forwards them to the matched upstream. Verify by reading the request log to confirm the header reached the provider. [REQUIRES-VERIFICATION] for the current list o… | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Portkey
- Ai Platform
- Unified Api And Gateway Proxy
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.