Auth Quotas Safety And Governance
Pinecone · Pinecone
Vector Database — Pinecone
Pinecone evals — Auth, Quotas, Safety & Governance (relift v3 InfraRed)
About Pinecone
Pinecone is a managed vector database for AI applications — serverless and pod-based indexes, namespaces for multi-tenant isolation, hybrid sparse-dense search, integrated inference (embed + rerank), and Pinecone Assistant for retrieval-augmented generation with citations.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator ships a SPA that calls /query directly with PINECONE_API_KEY in browser JavaScript. | Project-scoped API keys must never reach the browser — they grant full read/write across all namespaces in the project. Proxy through an operator-owned backend that scopes namespace by authenticated user. Rotate any key that touched a client bundle. | Pass / FailAi Platformcritical |
| 02 | Sustained traffic hits per-project QPS cap; queries start returning 429. | Honor retry-after header; apply exponential backoff with jitter; cap retry count. Treat 429 as backpressure — shed load upstream rather than burst-retry. Verify per-tier numerics against database-limits docs [REQUIRES-VERIFICATION]. | Pass / FailAi Platformhigh |
| 03 | Operator wants real-time visibility into project-level read/write/storage unit consumption. | Pinecone Console exposes usage dashboards; describe_index_stats gives per-namespace vector_count for storage estimation. Export to operator observability for alerting on approaching tier caps. Do not wait for 429 to discover quota pressure. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Pinecone
- Ai Platform
- Auth Quotas Safety And Governance
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.