Eval directory · Search & Knowledge
Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Hebbia AI products.
Hebbia is an AI platform that enables knowledge workers — primarily in finance and law — to perform complex research and analysis over large corpora of documents. Its retrieval and synthesis capabilities go beyond keyword search to reason across entire document sets.
Use the eval library for Hebbia
All 28 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.
Related in Search & Knowledge
All evals →Coverage map
2 areas · 28 graded scenarios
Every eval set is graded on
Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls
01
Smoke test for enterprise search & Q&A grounding and citations.
Mapped capabilities
3 scenarios
Example criterion: Hebbia delivers grounded, citation-faithful answers with strong resistance to unsupported hallucinations.
02
Evaluates Hebbia's Eval Factory Import — extraction accuracy, citation accuracy, and reasoning quality — across 25 test cases graded case by case by an LLM judge.
Mapped capabilities
25 scenarios
Example criterion: Hebbia responses follow required actions, avoid disallowed actions, and maintain risk-aware behavior.
Each eval pack tests Hebbia's public product surface — including Grounding Smoke V1 and Eval Factory Import V1 — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.
Pass/fail checks plus an LLM judge scoring 1–5 against each of the 28 Hebbia cases — from Eval Factory Import V1 (25 scenarios) down to the smallest pack — its own expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published to the Hebbia library.
The Hebbia eval library includes 28 graded test cases across 2 eval packs, the largest being Eval Factory Import V1 with 25 scenarios. Each case defines an input, expected behavior, and pass/fail criteria.
Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run all 2 Hebbia packs — Grounding Smoke V1 and Eval Factory Import V1 and the rest — against Hebbia or your own agent with your own data.
