Eval directory

Evals for Sybill

Eval coverage for Sybill, mapped from its public product surface.

About Sybill

Sybill is an AI sales assistant for B2B revenue teams that records and summarizes meetings, drafts follow-up emails, and answers questions about deals. It builds a shared 'context layer' across calls, emails, Slack, and CRM so teams can apply what wins to the next deal. It is sold in Free, Pro, Business, and Enterprise tiers with credit-based usage limits and integrates with CRMs such as Zoho.

Industry

AI sales assistant / revenue intelligence

Headquarters

2483 Old Middlefield Way, Ste 206, California - 94043

Website

www.sybill.ai

Use the eval library for Sybill

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Sybill?

6 scoring areas · 23 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Meeting Capture & Summarization

Recording, transcription, and summary/brief generation across virtual, dialer, and in-person meetings, including the Invisible Recorder and mobile capture paths.

“Al meeting summaries + briefs (20 meetings/month)” www.sybill.ai

Mapped capabilities

4 capabilities

Transcript-grounded summaries
Summary content traces to what was actually said; no invented attendees, commitments, or outcomes.
Customizable summary and brief formats
Respecting user-configured summary/brief templates and structure (Pro and above).
Pre-call briefs from prior context
Assembling a brief for an upcoming meeting from earlier calls, emails, and deal history.
In-person and mobile capture
Handling mobile app and in-person meeting capture, including degraded or partial audio conditions.

Follow-up Email Drafting

Post-call email drafts covering key points, action items, and next steps, with template, tone, and length controls before send.

“Yes, Sybill generates follow-up emails for each call.” www.sybill.ai

Mapped capabilities

4 capabilities

Action items and next steps extraction
Next steps in the draft correspond to commitments made on the call.
Template selection by deal stage
Applying the correct user-created template and regenerating on template switch.
Tone and length adjustment
Honoring requested tone/length changes without dropping substantive content.
Review-before-send behavior
Draft is surfaced for edit rather than sent autonomously.

Illustrative example

Input: After a 30-minute discovery call, the user generates a follow-up email using their "Discovery" template with tone set to concise and professional.
Expected behavior: Draft follows the selected template's structure, lists only next steps and action items stated on the call, and is presented for review and edit rather than sent.

Ask Sybill — Deal Q&A

Natural-language questions answered over the shared context layer spanning meetings, dialer calls, emails, Slack, and CRM.

“Over 850 revenue teams run on sybill” www.sybill.ai

Mapped capabilities

4 capabilities

Cross-deal pipeline questions
Questions like 'what's happening in my top 10 deals' answered from connected sources.
Source attribution
Answers indicate which call, email, or CRM record they came from.
Unknown and missing-context handling
Declining to answer when the connected sources don't contain the information.
Artifact generation on request
Producing deal artifacts such as a mutual action plan from available context.

Illustrative example

Input: On a Free-tier account with no CRM connected, the user asks Ask Sybill: "What's happening in my top 10 deals?"
Expected behavior: Answers from the meetings, dialer calls, and emails that are actually connected, and states that CRM context is not available on this plan rather than inferring pipeline stage or amounts.

CRM Sync & Integrations

Two-way flow with CRMs and workspace tools — Zoho CRM, Calendar, Mail, and Meetings — plus Slack, summary push, and programmatic access.

“CRM Autofill - 10 fields API + MCP access” www.sybill.ai

Mapped capabilities

4 capabilities

CRM autofill field mapping
Writing to the configured fields (10 on Business) with values supported by call or email evidence.
Meeting summaries pushed to CRM
Summary lands on the correct record and does not overwrite unrelated fields.
Two-way Slack behavior
Responding in configured Slack channels and reflecting actions taken back to the user.
API and MCP access
Programmatic access surface available on Business and above.

Plan Entitlements & Credit Limits

Tier gating across Free, Pro, Business, and Enterprise, plus weekly credit budgets and the caps attached to each plan.

Mapped capabilities

4 capabilities

Feature gating by tier
CRM integration in Ask Sybill, autofill, and API/MCP gated to Business and above.
Weekly credit accounting
Usage counted against the plan's weekly credit allowance (500 / 2500 / 5000 per user).
Free-tier summary cap
Behavior at and beyond the 20 meetings/month summary limit on Free.
Upgrade and limit messaging
Explaining which plan unlocks a blocked capability rather than silently failing.

Agentic Task Execution

Actions the assistant takes on the user's behalf — task execution, automated prompts in email and Slack, and deal workspace upkeep.

Mapped capabilities

3 capabilities

Automated prompt triggering
Proactive prompts delivered in Ask Sybill, email, and Slack or Teams at appropriate moments.
Write-action confirmation
Outbound or record-changing actions surfaced for approval before execution.
Deal workspace upkeep
Deal workspace and summaries stay consistent with the underlying calls and CRM state.

Coverage is mapped from Sybill's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Sybill test?+

The coverage map is generated from Sybill's own public product surface (AI sales assistant / revenue intelligence): 6 scoring areas — Meeting Capture & Summarization, Follow-up Email Drafting, and Ask Sybill — Deal Q&A, and more — spanning 23 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Sybill evals scored?+

Every case generated for Sybill — across Meeting Capture & Summarization and Follow-up Email Drafting and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Sybill library include?+

The full Sybill library is built on request. The coverage map spans 6 areas and 23 capabilities (for example, Transcript-grounded summaries and Customizable summary and brief formats under Meeting Capture & Summarization); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Sybill or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Sybill areas and set them up in a Corsac workspace, where you can run every test case against Sybill or your own agent with your own data.