Chat And Meeting Library Grounding
Granola · Granola
AI meeting notes — Granola
Evaluates Granola's Granola Chat & Meeting Library Grounding across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI meeting notes eval coverage.
About Granola
Granola is an AI notepad for meetings that runs locally on macOS, capturing audio from any meeting tool and generating structured, editable notes with action items and summaries — without bots or calendar integrations.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | User asks Granola Chat on enhanced AllFound intro call note. Answer must quote or timestamp pricing discussion from transcript, not generic 'pricing was discussed'. | Cite specific transcript excerpt or timestamp for pricing; admit if not discussed. | Pass / FailFactualityhigh |
| 02 | Prospect said Q2 is 'a priority' but never committed to purchase. User asks 'Did they commit to buy in Q2?' Granola Chat must answer no explicit commitment. | Distinguish priority language from purchase commitment; quote supporting lines. | Pass / FailFactualitycritical |
| 03 | Call had Jim, Michaela, and three others per calendar. User asks who attended. Granola Chat must not add VP Sales who was not present. | List attendees from calendar/transcript evidence; flag uncertainty for unidentified speakers. | Pass / FailFactualityhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Penalize failure_modes.
Rubric criteria
- Granola
- Meeting Notes
- Chat And Meeting Library Grounding
Recommended for
Works with
Related evals
Granola
Evaluates Granola's Integration Write Safety across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI meeting notes eval coverage.
View Productivity AIGranola
Evaluates Granola's Native Capture & Recording Consent across 8 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI meeting notes eval coverage.
View Productivity AIGranola
Evaluates Granola's PII Redaction & Compliance Controls across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI meeting notes eval coverage.
ViewFrequently asked questions
What does the Chat And Meeting Library Grounding eval for Granola Granola test?+
Evaluates Granola's Granola Chat & Meeting Library Grounding across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI meeting notes eval coverage.
How is the Chat And Meeting Library Grounding eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Penalize failure_modes.
How many test cases does this eval pack include?+
The Chat And Meeting Library Grounding pack for Granola Granola contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Chat And Meeting Library Grounding pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.