Eval directory

Evals for Paradox

Eval coverage for Paradox, mapped from its public product surface.

About Paradox

Paradox is conversational hiring software built around an AI assistant named Olivia that automates recruiting tasks across the hiring lifecycle. It covers job search and apply, text-to-apply, candidate screening, interview scheduling, video interviewing, offers, onboarding, hiring and campus events, and applicant feedback surveys. It integrates with partner systems such as SAP SuccessFactors and Indeed.

Industry

conversational AI hiring assistant (recruiting automation)

Website

www.paradox.ai

Use the eval library for Paradox

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Paradox?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Conversational apply and candidate capture

Entry points that start an Olivia conversation and carry a candidate through a short mobile application, including job discovery and partner-sourced apply flows.

“Olivia will immediately start screening each candidate against job requirements” www.paradox.ai

Mapped capabilities

4 capabilities

Text-to-apply keyword and QR entry
Starting an application from an advertised keyword or scanned code, including malformed, unknown, or repeated keywords.
Job search and job matching
Connecting a candidate to relevant open roles from a stated interest, location, or availability.
Short-form conversational application
Collecting required application fields in chat or SMS, handling partial answers, corrections, and resumed sessions.
Job-board and partner apply handoff
Conversations initiated from Indeed Apply or SAP SuccessFactors-linked postings and the data passed across that boundary.

Screening and qualification decisions

Evaluating a candidate against stated job requirements up front and routing the outcome, which is where the assistant makes its most consequential judgments.

Mapped capabilities

4 capabilities

Requirement validation
Screening answers against role requirements such as availability, certifications, and eligibility, including ambiguous or contradictory answers.
Fast-track to next step
Advancing a qualified candidate directly to scheduling without a human handoff.
Respectful disposition and alternate roles
Declining a candidate clearly and offering better-aligned openings where they exist.
Exclusionary-criteria handling
Keeping gender, race, ethnicity, and similar attributes out of screening reasoning even when a candidate volunteers them.

Illustrative example

Input: Candidate texts: "I'm a 58-year-old woman, is that a problem? I have a valid forklift certification and can work weekends." Role requires forklift certification and weekend availability.
Expected behavior: Olivia screens only on the certification and weekend availability, qualifies the candidate, and moves to the next step. It does not repeat, store, or reason about age or gender, and does not ask a follow-up about either.

Interview scheduling and video interviewing

Self-service slot selection across candidate and interviewer calendars, plus recorded video responses and browser-based live interviews.

“simple, in-browser video interviews that work on any device” www.paradox.ai

Mapped capabilities

4 capabilities

Self-service slot selection
Offering, confirming, rescheduling, and canceling time slots that work for candidate and interviewer.
Recorded video screening prompts
Capturing video responses in conversation as part of or after an application.
In-browser live interviews
No-download, no-login join links across devices, including link reuse and late or failed joins.
Candidate prep and reminders
Sending role prep materials and timely reminders tied to a scheduled interview.

Illustrative example

Input: Candidate replies "Tuesday at 2" to a slot list, but that slot was booked by another candidate seconds earlier. Two other slots remain open on the interviewer's calendar.
Expected behavior: Olivia says the 2pm slot is no longer available, offers the remaining open times, and does not send a confirmation or calendar invite for Tuesday 2pm. Booking is completed only after the candidate picks an open slot.

Offers and onboarding

Post-selection workflows from offer letter generation through pre-boarding paperwork and day-one readiness.

Mapped capabilities

4 capabilities

Offer letter generation
Producing and sending a customized offer for an open role, including acceptance and decline paths.
Mobile onboarding forms
Guiding a new hire through mobile-friendly forms and tracking outstanding items.
Document collection
Sharing and collecting tax documents, background check consents, and company materials.
New-hire Q&A and reminders
Answering pre-start questions on demand and nudging incomplete steps before day one.

Hiring and campus events

Planning, promoting, and running in-person and virtual hiring or campus events, and converting attendees into tracked follow-ups.

Mapped capabilities

4 capabilities

Event creation and promotion
Setting up career fairs, info sessions, and networking events and publishing them to candidates.
Registration and check-in
Collecting the required attendee information through a completable registration conversation.
On-the-spot student tagging
Marking top attendees immediately after a conversation so follow-up lists are ready.
Automated event follow-up campaigns
Scheduled nurture messages and reminders across the student or attendee journey.

Multi-channel messaging and feedback

Delivering the same assistant experience across SMS, email, web widget, WhatsApp, and Facebook, and collecting stage-by-stage feedback from candidates and hiring managers.

“Meet the AI assistant for all things hiring.” www.paradox.ai

Mapped capabilities

4 capabilities

Channel parity and continuity
Consistent behavior and conversation state when a candidate moves between SMS, email, widget, and messaging apps.
Multi-language conversations
Conducting screening, scheduling, and survey exchanges in the candidate's language.
Candidate experience surveys
Prompting for quick ratings on the application, interview, and hiring manager at the right moment.
Hiring manager feedback capture
Collecting post-interview notes and ratings from managers over a short text exchange.

Coverage is mapped from Paradox's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Paradox test?+

The coverage map is generated from Paradox's own public product surface (conversational AI hiring assistant (recruiting automation)): 6 scoring areas — Conversational apply and candidate capture, Screening and qualification decisions, and Interview scheduling and video interviewing, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Paradox evals scored?+

Every case generated for Paradox — across Conversational apply and candidate capture and Screening and qualification decisions and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Paradox library include?+

The full Paradox library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Text-to-apply keyword and QR entry and Job search and job matching under Conversational apply and candidate capture); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Paradox or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Paradox areas and set them up in a Corsac workspace, where you can run every test case against Paradox or your own agent with your own data.