Eval directory

Evals for Zoom

Eval coverage for Zoom, mapped from its public product surface.

About Zoom

Zoom is an AI-first work platform that unifies meetings, phone, chat, mail and calendar, webinars, whiteboards, and file sharing in a single suite. It layers AI assistants — ZoomMate, My Notes, AI Companion, and Virtual Agent — across those surfaces to summarize meetings, automate follow-ups, and generate assets like decks and documents. Adjacent products extend it into customer support (Contact Center), employee experience (Workvivo), and small-business operations (Bonsai).

Industry

AI-first unified communications and collaboration platform (UCaaS)

Website

www.zoom.com

Use the eval library for Zoom

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Zoom?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Meetings and unified communications core

The real-time and asynchronous communication substrate — video meetings, cloud phone, chat, and mail/calendar/scheduling — presented as a single UCaaS surface rather than separate tools.

“Phone, chat, email, SMS, social, and video unified in a single view.” www.zoom.com

Mapped capabilities

4 capabilities

Video meetings and hybrid room experiences
Hosting and joining meetings, Zoom Rooms for in-person plus remote participants, and media quality options such as the Enhanced Media add-on.
Cloud phone and cross-platform calling
Zoom Phone calling behavior, including placing and receiving Zoom Phone calls from within the Microsoft Teams integration.
Team chat and unified inbox context
Instant messaging across workplace apps and the unified view spanning phone, chat, email, SMS, social, and video.
Mail, calendar, and external scheduling
Working alongside an existing mail/calendar service and sharing availability for external booking via Scheduler.

AI assistants and meeting intelligence

The AI layer marketed across the suite — My Notes, AI Companion, and ZoomMate — covering transcription, summarization, follow-up automation, cross-app retrieval, and agentic workflows.

Mapped capabilities

4 capabilities

Note taking and meeting summarization
My Notes transcription and note taking across platforms, including mobile capture of in-person and on-the-go conversations.
Follow-up and recurring asset automation
Automating next steps after a call and Auto Writing's recurring recaps, reports, and proposals for a configured meeting series.
Agentic search over connected systems
Ask AI Companion questions spanning connectors such as Salesforce, Google Drive, ServiceNow, Box, Workday, Confluence, SharePoint, and Zendesk under the Custom AI Companion add-on.
Workflow orchestration and external agent access
AI Companion Workflow nodes including Deep Reasoning, and exposing meeting intelligence to external agents through the MCP integration with Claude.

Illustrative example

Input: A 30-minute transcript where one participant says "I'll send the pricing sheet by Friday" and no budget is ever discussed. Ask My Notes to summarize and list action items.
Expected behavior: The summary attributes the pricing-sheet follow-up to the participant who committed to it with the Friday due date, and includes no budget action item or owner who was not in the meeting.

Content creation and collaboration workspace

Where meeting output becomes durable work product: AI-first documents, whiteboards, clips, generated decks, tasks, and the Hub that stores and surfaces all of it.

Mapped capabilities

4 capabilities

AI-first documents and whiteboards
Zoom Canvas real-time collaborative documents and Whiteboard's canvas plus advanced diagramming.
Deck and video asset generation
Zoom Slides generating a deck from a prompt, a Canvas, or meeting content; Clips recording and editing, including custom AI avatars delivering a presentation script.
Content hub organization and retrieval
Hub folders, filters, and search across recordings, summaries, canvases, whiteboards, and clips, plus AI file summaries without opening the file.
Task capture and completion
Surfacing, managing, and completing tasks across Zoom Workplace with ZoomMate.

Webinars and virtual events

The one-to-many broadcast surface spanning webinars, virtual and hybrid events, on-demand video, production tooling, and the analytics that tie events to pipeline.

“Turn a single prompt into a production-ready virtual agent in minutes, no scripting or flow-building required.” www.zoom.com

Mapped capabilities

4 capabilities

Webinar setup, branding, and production
Custom branding and production tooling for events ranging from simple webinars to full broadcasts without technical skills.
Virtual and hybrid attendee experience
Lobbies, expo floors, networking hubs, and bridging in-person with remote attendees via mobile chat and on-screen video.
On-demand libraries and engagement analytics
Turning recordings into searchable on-demand libraries, auto-generating social clips, and tracking viewer engagement.
Training delivery and certification
Setting completion requirements for live training sessions and automatically issuing certificates to participants.

Customer support automation

Contact Center and Virtual Agent — self-service deflection, agent assist, and the tooling used to build and tune automated support experiences.

“Virtual Agent handles multi-intent questions, so human agents can focus on high-value cases.” www.zoom.com

Mapped capabilities

4 capabilities

Virtual Agent multi-intent handling and self-service
Handling multi-intent questions, delivering instant answers and proactive resolutions, and routing high-value cases to human agents.
Agent build tooling and domain vocabulary
Agent Architect turning a prompt into a production-ready virtual agent, and ZVA Custom Dictionary for business terms and phonetic spellings.
Live agent assist
Real-time suggestions, key action highlights, and task automation for human agents during interactions.
Support analytics and CRM context
CRM integration, real-time analytics, and conversation insights that surface trends across support interactions.

Illustrative example

Input: Customer asks: "Where is my refund for order 4471, and can you also change the email on my account to sam@example.com?"
Expected behavior: The agent addresses both requests rather than only the first, giving refund status for order 4471 and either completing or explicitly handing off the email change, without silently dropping either one.

Adjacent business platforms: Workvivo and Bonsai

Products extending Zoom beyond collaboration — employee experience for internal audiences and an all-in-one operations platform for small businesses.

Mapped capabilities

4 capabilities

Employee communication and engagement
Workvivo activity and social feeds, digital signage, livestream, podcast, newsletters, shout-outs, and community spaces reaching desk and frontline staff.
Intranet, search, and sentiment insight
Workvivo documents, wiki pages, app launcher, search, and 100+ integrations, plus surveys, polls, dashboards, and read-and-acknowledge analytics.
Client lifecycle and CRM automation
Bonsai CRM and pipeline with ZoomMate updating records after calls, email/calendar sync, and syncing Zoom meeting recordings, transcripts, and AI summaries to contacts.
Client-facing documents and payment flow
Bonsai proposals, agreements with e-signature and deposit collection, branded intake forms, and the client portal spanning first call to final payment.

Coverage is mapped from Zoom's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Zoom test?+

The coverage map is generated from Zoom's own public product surface (AI-first unified communications and collaboration platform (UCaaS)): 6 scoring areas — Meetings and unified communications core, AI assistants and meeting intelligence, and Content creation and collaboration workspace, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Zoom evals scored?+

Every case generated for Zoom — across Meetings and unified communications core and AI assistants and meeting intelligence and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Zoom library include?+

The full Zoom library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Video meetings and hybrid room experiences and Cloud phone and cross-platform calling under Meetings and unified communications core); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Zoom or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Zoom areas and set them up in a Corsac workspace, where you can run every test case against Zoom or your own agent with your own data.