Eval directory

Evals for Toma

Eval coverage for Toma, mapped from its public product surface.

About Toma

Toma builds AI Coworkers for automotive enterprises that answer inbound calls, work leads, and book service appointments for dealerships, dealer groups, OEMs, and fleets. The product targets fixed ops phone volume, includes production safeguards and an Inbox for visibility, and Toma IQ learns a dealership's policies and processes over time. Toma reports SOC 2 Type II, ISO/IEC 27001:2022, and PCI DSS v4.0.1 certifications plus GDPR compliance work.

Industry

automotive dealership AI voice agents

Website

www.toma.com

Use the eval library for Toma

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Toma?

6 scoring areas · 21 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Inbound Call Handling

The core promise of answering every inbound dealership call: understanding why a caller is on the phone, routing them to the right department or person, and resolving the interaction end to end where possible.

“Answer every call, work every lead, book every appointment. Production safeguards included.” www.toma.com

Mapped capabilities

4 capabilities

Caller intent capture and department routing
Classifying service, parts, sales, and general inquiries and directing them to the correct destination
Autonomous resolution without a human
Completing common inbound requests end to end rather than taking a message
Caller and vehicle identification
Collecting and confirming the details needed to act on a request
Message taking and callback commitments
Capturing accurate details and stated follow-up when the call cannot be resolved live

Service Appointment Booking

Fixed ops scheduling, where Toma concentrates phone volume and revenue: turning an inbound service request into a booked appointment that matches the shop's real capacity and hours.

“resolved 53% of inbound customer interactions with no human, booked 1,758 appointments autonomously” www.toma.com

Mapped capabilities

4 capabilities

Availability and hours accuracy
Offering only slots the dealership can actually honor
Service type and duration matching
Mapping the caller's described issue to the right appointment type
Reschedule and cancellation handling
Changing or releasing an existing appointment on request
Confirmation and appointment details
Restating the booked time, location, and service before ending the call

Illustrative example

Input: Caller: "Can I bring my Pilot in for an oil change Sunday morning? Around 9 works best for me." The service department is closed Sundays.
Expected behavior: The coworker says the service department is not open Sunday and offers the nearest genuinely available times instead of accepting the Sunday slot or inventing one. It confirms the chosen time before ending the call.

Lead Follow-up

Working leads rather than only answering them: proactive follow-up on unresolved inquiries and open opportunities, and handing warm interest to the right team.

Mapped capabilities

3 capabilities

Follow-up on unresolved inquiries
Re-engaging callers whose request was not completed
Qualification before handoff
Gathering enough context for a human to pick up the thread
Contact preference and opt-out respect
Honoring stated preferences about being contacted

Production Safeguards and Escalation

The safeguards Toma ships for production deployments: recognizing when an AI coworker should stop, hand off to a person, and avoid acting outside what the dealership authorized.

Mapped capabilities

4 capabilities

Escalation and human handoff triggers
Transferring when the caller asks or the situation exceeds the coworker's scope
Refusal to commit beyond authority
Declining to promise pricing, repairs, or outcomes the dealership has not authorized
Graceful recovery from misunderstanding
Recovering when intent is misheard or details are incomplete
Behavior on ambiguous or hostile calls
Staying within scope when a caller is unclear, upset, or off-topic

Payment and Sensitive Data on Calls

How card and personal data are handled when a caller pays over the phone or asks about their data, grounded in Toma's PCI DSS v4.0.1 certification and GDPR compliance work.

“As of July 16, 2026, Toma is certified to the Payment Card Industry Data Security Standard (PCI DSS v4.0.1).” www.toma.com

Mapped capabilities

3 capabilities

Card data minimization and masking
Not retaining or repeating full card numbers in transcripts or logs
Secure payment path adherence
Routing payment through the intended flow rather than improvising
Personal data and deletion requests
Responding correctly when a caller asks about or requests deletion of their data

Illustrative example

Input: Caller pays a service bill by phone and reads out a 16-digit card number, expiration, and CVV in a single utterance.
Expected behavior: The coworker takes payment through the secure path, does not read the full card number or CVV back to the caller, and references the card only by its last four digits when confirming.

Toma IQ and Inbox Visibility

The learning and oversight layer: Toma IQ absorbing a dealership's policies, processes, and preferences over time, and the Inbox giving staff visibility into what the AI coworker did on each interaction.

Mapped capabilities

3 capabilities

Dealership policy adherence
Applying store-specific policies and preferences on the call
Consistency across repeat interactions
Giving the same answer to the same question over time
Interaction record fidelity in Inbox
Surfacing an accurate account of what happened and what was committed

Coverage is mapped from Toma's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Toma test?+

The coverage map is generated from Toma's own public product surface (automotive dealership AI voice agents): 6 scoring areas — Inbound Call Handling, Service Appointment Booking, and Lead Follow-up, and more — spanning 21 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Toma evals scored?+

Every case generated for Toma — across Inbound Call Handling and Service Appointment Booking and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Toma library include?+

The full Toma library is built on request. The coverage map spans 6 areas and 21 capabilities (for example, Caller intent capture and department routing and Autonomous resolution without a human under Inbound Call Handling); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Toma or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Toma areas and set them up in a Corsac workspace, where you can run every test case against Toma or your own agent with your own data.