Eval directory
Evals for Exa Labs
7 evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Exa Labs AI products.
About Exa Labs
Exa (formerly Metaphor) is a neural search API that understands the meaning of queries rather than matching keywords — returning the most relevant URLs and content from the web for any semantic question. Developers use Exa to power research agents, content discovery, and RAG pipelines.
Available eval packs for Exa Labs
7 packs ready to run.
Contents Freshness Cache Semantics
Exa Labs evals — Contents Freshness & Cache Semantics (relift v3)
Corpus Safety Prompt Injection
Prompt InjectionExa Labs evals — Corpus Safety & Prompt Injection (relift v3)
Filters Domains Date Controls
Exa Labs evals — Filters Domains & Date Controls (relift v3)
Find Similar Semantic Neighbors
Exa Labs evals — Find Similar & Semantic Neighbors (relift v3)
Governance Pii Copyright Rate Limits
PII LeakageExa Labs evals — Governance PII Copyright & Rate Limits (relift v3)
Highlights Summary Synthesis Grounding
Answer RelevanceExa Labs evals — Highlights Summary & Synthesis Grounding (relift v3)
Search Mode Selection Query Routing
Exa Labs evals — Search Mode Selection & Query Routing (relift v3)
Why eval Exa Labs AI
Exa Labs's AI features ship behind brand promises about accuracy, safety, and reliability. Buyers and integrators need to know those promises hold up under adversarial prompts, edge-case workflows, and the long tail of real customer inputs — not just the demo path.
The Corsac eval library for Exa Labs measures four dimensions teams care about most when deploying search & knowledge agents:
- Adversarial robustness — does the agent resist prompt injection, jailbreaks, and social-engineering attempts?
- Workflow quality— does it complete the task buyers were shown in the demo, on inputs that don't look like the demo?
- Safety gates — does it escalate or refuse when it should, and only then?
- Operator quality — does it preserve analyst trust by surfacing the right context at the right time?
Every eval pack above is hand-authored against Exa Labs's public product surface and runnable in Corsac with your own data.