Eval directory

Evals for Upfirst

Eval coverage for Upfirst, mapped from its public product surface.

About Upfirst

Upfirst is an AI answering service and virtual receptionist that answers inbound calls for small businesses so calls and leads aren't missed. It takes messages, answers common questions, books appointments into calendars, transfers calls to staff, and sends call summaries, recordings, and transcripts. Plans start at $24.95/month (or $20/month billed annually) with per-additional-call pricing and a free trial requiring no credit card.

Industry

AI phone answering service / virtual receptionist for small businesses

Website

upfirst.ai

Use the eval library for Upfirst

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Upfirst?

6 scoring areas · 23 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Inbound Call Answering & Message Capture

Answering the call and collecting exactly the caller details the business configured, so nothing is missed and nothing extra is invented.

“The key details from every call, texted or emailed the moment it ends.” upfirst.ai

Mapped capabilities

4 capabilities

Greeting and caller intent capture
Opens the call, identifies the business, and establishes what the caller needs.
Configured message fields
Collects only the fields the business chose to collect from each caller.
Contact detail accuracy
Captures and confirms name, callback number, and reason back to the caller.
Spam and junk call handling
Recognizes and blocks spam calls rather than capturing them as leads.

Taught Answers & Question Handling

Answering common customer questions from business-supplied knowledge, and behaving safely when the answer was never taught.

“Upfirst is the AI answering service that keeps your phone covered so you never miss a lead” upfirst.ai

Mapped capabilities

3 capabilities

Answering from configured FAQs
Returns the business's own taught answer to a common question.
Unknown-question behavior
Declines to fabricate when a question falls outside what was taught.
Handoff to message or transfer
Converts an unanswerable question into a captured message or a routed call.

Illustrative example

Input: Caller: "How much do you charge for a crown, and does that include the temporary?" The business has taught hours and location, but no pricing information.
Expected behavior: The receptionist says it does not have pricing on hand rather than guessing a number, then captures a message with the caller's name, number, and question, or offers a transfer to staff.

Appointment Scheduling

Turning an inbound call into a booking, either by sending a scheduling link or by collecting a preferred date and time.

Mapped capabilities

4 capabilities

Preferred date/time collection
Asks for and confirms the caller's requested slot.
Calendar link delivery
Sends the booking link to the caller during the call.
Calendar integrations
Books into the connected calendar (Google, Outlook, Clio, Acuity, Calendly, and more).
Booking confirmation to caller
States clearly what was and was not booked before ending the call.

Illustrative example

Input: Caller: "Hi, I need to bring my dog in for a checkup. Is Thursday morning any good? My name is Dana Reyes, 555-0142."
Expected behavior: The receptionist confirms a specific date and time for Thursday morning, repeats the caller's name and callback number, and states plainly that the appointment is booked. If no slot is available, it offers an alternative or sends a scheduling link instead.

Call Routing & Live Transfer

Sending the right calls to the right person, and covering the calls that should not be transferred.

“Route the right calls to you or your team with a live, warm handoff.” upfirst.ai

Mapped capabilities

4 capabilities

Route-to-person rules
Directs a call to the staff member the business designated for that call type.
Warm handoff
Introduces the caller to the person receiving the live transfer.
Failed-transfer fallback
Falls back to message capture when the transfer target is unavailable.
Coverage mode selection
Applies the configured mode: every call, overflow only, or after hours.

Post-Call Delivery & Integrations

What the business receives after the call ends, and where that data lands downstream.

Mapped capabilities

4 capabilities

Instant call summaries
Texts or emails the key details the moment the call ends.
Recording and transcript fidelity
Transcript and recording reflect what was actually said on the call.
Zapier / CRM push
Sends caller details into the connected CRM or downstream app.
Delivery channel selection
Honors the business's choice of text versus email delivery.

Caller Experience & Continuity

How the receptionist sounds and adapts across languages, voices, mid-call texting, and returning callers.

“Recognizes returning callers and recalls past calls, so nobody repeats themselves.” upfirst.ai

Mapped capabilities

4 capabilities

Automatic language switching
Switches among supported languages when the caller does, then translates the transcript.
Voice and persona selection
Uses the selected voice, gender, and accent from the voice directory.
In-call texting
Sends a scheduling link, pricing, or directions by text mid-call.
Returning-caller memory
Recognizes a repeat caller and recalls prior calls so details are not re-asked.

Coverage is mapped from Upfirst's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Upfirst test?+

The coverage map is generated from Upfirst's own public product surface (AI phone answering service / virtual receptionist for small businesses): 6 scoring areas — Inbound Call Answering & Message Capture, Taught Answers & Question Handling, and Appointment Scheduling, and more — spanning 23 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Upfirst evals scored?+

Every case generated for Upfirst — across Inbound Call Answering & Message Capture and Taught Answers & Question Handling and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Upfirst library include?+

The full Upfirst library is built on request. The coverage map spans 6 areas and 23 capabilities (for example, Greeting and caller intent capture and Configured message fields under Inbound Call Answering & Message Capture); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Upfirst or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Upfirst areas and set them up in a Corsac workspace, where you can run every test case against Upfirst or your own agent with your own data.