Eval directory
Evals for Braintrust
3 evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Braintrust AI products.
About Braintrust
Braintrust is an AI evaluation and observability platform for building reliable AI products. It combines offline evaluation runs (a task function graded over a dataset by one or more scorers), production tracing and logging, online scoring of live traffic, no-code Playgrounds for iterating on prompts/models/scorers, and Brainstore — a purpose-built log store for fast querying of AI traces. Scorers can be heuristic code, LLM-as-a-judge, or pre-built Autoevals, invocable via a Functions API; SDKs cover Python, TypeScript, Go, Ruby, and C#. Enterprise controls include RBAC, SSO, encrypted secrets, SOC 2 Type II, and a hybrid control/data-plane deployment that keeps customer AI data in their own VPC. [REQUIRES-VERIFICATION] employee count and headquarters.
Employees
[REQUIRES-VERIFICATION]
Industry
AI Evaluation & Observability
Headquarters
San Francisco, CA [REQUIRES-VERIFICATION]
Website
www.braintrust.devAvailable eval packs for Braintrust
3 packs ready to run.
Evaluation Runs Eval Correctness
Correctness30 graded scenarios covering edge cases, failure modes, and quality checks.
Scorer Authoring Calibration
23 graded scenarios covering edge cases, failure modes, and quality checks.
Tracing Span Fidelity
3 graded scenarios covering edge cases, failure modes, and quality checks.
Why eval Braintrust AI
Braintrust's AI features ship behind brand promises about accuracy, safety, and reliability. Buyers and integrators need to know those promises hold up under adversarial prompts, edge-case workflows, and the long tail of real customer inputs — not just the demo path.
The Corsac eval library for Braintrust measures four dimensions teams care about most when deploying medical & clinical ai agents:
- Adversarial robustness — does the agent resist prompt injection, jailbreaks, and social-engineering attempts?
- Workflow quality— does it complete the task buyers were shown in the demo, on inputs that don't look like the demo?
- Safety gates — does it escalate or refuse when it should, and only then?
- Operator quality — does it preserve analyst trust by surfacing the right context at the right time?
Every eval pack above is hand-authored against Braintrust's public product surface and runnable in Corsac with your own data.