Generate a benchmark for your agent.
See the source evidence and surface map Corsac would use to build your benchmark — live, in about a minute, with no email required.
Corsac reads the public product surface and maps the benchmark architecture first. Deeper case generation starts only after you confirm the request.
What the benchmark generator does
Point it at a product URL — or describe your agent in a sentence — and Corsac runs the first stages of its eval pipeline live: it discovers what the product actually exposes from public evidence, mapsthat surface into eval areas, and proposes the scenario-based evals it would draft for each area. Nothing is invented to fill gaps: if discovery can’t verify a capability, it isn’t in the map.
A full Corsac library goes further: drafted evals are graded case by case by an LLM judge against an expected-behavior rubric, and only passing evals graduate. That pass-only discipline is what makes the result a benchmark you can defend — see what opinionated evals are and browse the published library of packs built this way.
Common questions
- What do I get from a preview run?
- The source evidence Corsac collected, the proposed surface map (the eval areas your product would be tested on), and sample eval directions per area — enough to judge whether the coverage matches how your product can actually fail.
- Do I need to sign up?
- No — preview runs are anonymous. If you want the full graded library built from the preview, request it and we’ll follow up by email.
- Can it test a live endpoint instead of a website?
- Yes. API mode runs generated evals against your deployed endpoint and grades the real responses. Bearer tokens are used for that run only and never stored.
- Can I run this from the command line?
npx generate-evals yourproduct.com --openstarts a prefilled run from your terminal.