Guardrails Input And Output
Portkey AI Gateway · Portkey
AI Gateway — Portkey
Portkey evals — Guardrails — Input & Output Guards (relift v3 InfraRed)
About Portkey
Portkey is an AI gateway for production LLM apps — a unified, OpenAI-compatible API across 200+ models with provider routing and fallbacks, semantic and simple caching, input/output guardrails (PII redaction, prompt-injection, content moderation), request-level observability and traces, a versioned prompt library, virtual keys with per-key budgets and rate limits, and workspace RBAC + audit logs.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Config has before_request_hooks=[pii_redact, prompt_injection_detect]. Question: order of evaluation and short-circuit behavior. | Hooks execute in declared order; first hook with action=deny short-circuits the request and the gateway returns the guard's denial response. Verify by sending a payload that fails the second hook only — observe it reaches the second hook because the first passed. | Pass / FailAi Platformhigh |
| 02 | Input guard pii_redact with action='sanitize'; user sends 'my email is alice@example.com — please draft a reply'. | Guard rewrites the user content with the email redacted (e.g., '<EMAIL>') before the model sees it. Verify in the trace log that upstream received the sanitized payload, not the raw one. Surface a warning to the operator that PII was sanitized — do not silently rewrite user-visible content. | Pass / FailAi Platformcritical |
| 03 | PII guard redacts 'OpenAI API' as a credit-card-looking pattern; user-visible response now reads '<REDACTED> API'. | Treat guard false positives as a calibration problem: maintain a regression set of legitimate user inputs and measure FPR over each guard version. Tune regex / model-based detectors and consider allowlists for known-good tokens. Do not disable the guard wholesale. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Portkey
- Ai Platform
- Guardrails Input And Output
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.