Corsac
Use Corsac
CorsacDocs / MCPpublic preview

Bring Corsac benchmarks into your AI assistant

Search public eval packs and map benchmark coverage for an AI product from the assistant where you already work. Corsac speaks MCP, so you ask in natural language—not JSON-RPC.

Connect Corsac once

The hosted server is available at https://api.corsac.ai/mcp. No Corsac account is required for the public preview.

Claude Code

claude mcp add --transport http --scope user corsac https://api.corsac.ai/mcp

Codex

codex mcp add corsac --url https://api.corsac.ai/mcp
The name corsac is your local nickname for the connection. The scope tells Claude Code to make that connection available across your projects. You do this once; afterward you talk to Corsac through normal prompts.

Ask for the outcome, not the plumbing

Try one of these prompts after connecting:

  • “Find Corsac eval packs for customer-support agents. Don’t start a preview.”
  • “Search Corsac for Decagon benchmarks and explain the coverage.”
  • “Create a benchmark preview for https://example.com. Search the library first, ask before starting it, then poll until it finishes.”

Your assistant chooses and calls the MCP tools. You should not need to construct request payloads yourself.

Three focused tools

search_eval_libraryRead only

Search public pack metadata by company, product, risk, or workflow. Returns five matches by default and at most ten.

start_benchmark_previewStarts a job

Start one bounded, URL-based preview. Your client should ask for approval before invoking it.

get_benchmark_statusRead only

Poll a temporary run and return its evidence, proposed coverage areas, and up to two illustrative examples.

A preview is coverage—not a score

Corsac reads public product evidence and proposes what a useful benchmark should test. It does not run those cases against the product, observe its private behavior, or claim that the product passed or failed.

Say “proposed benchmark coverage grounded in public evidence.” Do not describe a preview as measured model or agent performance.

To measure performance, a developer still needs to run the resulting eval cases against their own agent or model endpoint. Open the benchmark generator

Public data and bounded usage

  • Library search exposes metadata, not vaulted rubrics, answer keys, private runs, or captured artifacts.
  • Anonymous previews accept public product URLs. Free-text product descriptions and private dataset uploads are not enabled on the anonymous surface.
  • Preview starts are rate-limited and globally bounded to protect crawl and generation capacity.
  • Temporary run status may expire; start a new preview if a run is no longer available.

See the privacy policy for how submitted URLs and service metadata are handled.

Troubleshooting

Rate-limit message: wait for the stated cooldown. Opening another terminal on the same network will not create a new quota.

Discovery failed: the target site may block automated access or may not expose enough public content. The failure does not indicate anything about the product’s quality.

Permission declined: no preview starts. Ask again when you are ready and approve exactly one tool invocation.

Still stuck? Contact Corsac support