Eval directory

Evals for PitchMonster

Eval coverage for PitchMonster, mapped from its public product surface.

About PitchMonster

PitchMonster is a focused AI sales role-play platform where reps rehearse realistic buyer conversations before live calls and debrief with an AI coach afterward. It combines AI role-play, real call analysis and scoring, and AI coaching feedback built around a team's ICP, products, and objections. It is sold per seat via a demo-and-quote process and targets sales, CS, enablement, and partner-training use cases.

Industry

AI sales role-play and coaching software

Website

www.pitchmonster.io

Use the eval library for PitchMonster

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for PitchMonster?

6 scoring areas · 22 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

AI Role-Play Buyer Simulation

The core practice surface: reps rehearse realistic conversations with AI buyers that adapt, push back, and behave like the team's actual ICP across call types, modes, and languages.

“PitchMonster helps sales teams practice real conversations, analyze live calls, and get AI coaching” www.pitchmonster.io

Mapped capabilities

4 capabilities

Persona fidelity and adaptive pushback
AI buyer holds its assigned persona (e.g. VP of Sales, Procurement Lead, General Practitioner, Homeowner), objects and pushes back rather than conceding, and does not break into coach or narrator voice mid-session.
Call-type and mode coverage
Cold call, warm call, discovery, and demo role-plays behave distinctly across voice, chat, and video practice modes.
Multi-persona and gatekeeper scenarios
Sessions involving more than one buyer-side participant, including gatekeeper handling and handoffs between roles.
Multilingual role-play
Practice conversations conducted across the supported 29 languages without persona or objection behavior degrading.

Illustrative example

Input: Cold-call role-play, Procurement Lead persona. The buyer requires a security review, a signed DPA, and SOC 2 proof. The rep replies "I'll send something over later" and pivots to booking a follow-up.
Expected behavior: The AI buyer stays in the Procurement Lead persona and restates the specific unmet requirements rather than accepting the vague promise or agreeing to the follow-up. It does not drop into coach, narrator, or assistant voice mid-session.

AI Coaching, Scoring & Scorecards

The debrief surface: coaching feedback on every run, custom scorecards, and scoring built on the team's chosen methodology, plus detection of objections and questions in the conversation.

“Scoring built on your methodology (MEDDIC, SPIN, Sandler)” www.pitchmonster.io

Mapped capabilities

4 capabilities

Custom scorecard application
Team-defined scorecards are applied consistently to a completed run, with per-criterion outcomes tied to what actually happened in the session.
Methodology-based scoring
Scoring against MEDDIC, SPIN, or Sandler reflects the named framework's elements rather than generic sales feedback.
Objection and question detection
Objections raised and questions asked during a session are identified and attributed to the correct speaker and moment.
Socratic coaching debrief
Post-session AI Coach feedback is grounded in transcript evidence and coaches through questioning rather than asserting unobserved performance.

Illustrative example

Input: A completed discovery role-play transcript in which the rep never asks about budget, metrics, or the decision process, submitted for AI coaching feedback scored on MEDDIC.
Expected behavior: The scorecard marks Metrics, Economic buyer, and Decision process as not covered, and the coaching feedback cites specific transcript moments. It gives no positive credit for skills the rep did not demonstrate in the session.

Real Call Analysis

Analysis of live customer calls, not just simulations: uploading real recordings, transcribing them in full, and scoring them with the same scorecards used for practice.

Mapped capabilities

3 capabilities

Call upload and transcription
Uploaded real calls are transcribed in full with speaker separation.
Scoring real calls against team scorecards
The same custom and methodology-based scorecards apply to real recorded calls.
Practice-to-live continuity
Findings from real calls and from role-play sessions are expressed consistently so gaps can be traced across both.

Scenario Authoring from ICP, Products & Objections

Configuration surface where enablement teams turn their own ICP, product set, and known objections into custom scenarios, alongside the ready-made practice library.

Mapped capabilities

3 capabilities

ICP-grounded scenario generation
Custom scenarios reflect the supplied ICP, products, and objection set rather than generic buyer archetypes.
Ready-made practice library
Out-of-the-box scenarios are usable before any customization is configured.
Fidelity to supplied product facts
Scenario content stays within the customer-provided product and objection material and does not invent unsupplied product claims.

Training Programs Across the Sales Cycle

The workflow surface for the named use cases: new hire ramp-up, post-training certification, low-performer upskilling, channel and reseller training, post-sale role-plays, and candidate screening.

Mapped capabilities

4 capabilities

New hire ramp and certification
Ramp and post-training certification paths surface where a rep falls short before a first live call.
Channel and partner standardization
Presentation quality is assessed consistently for partners the team does not directly manage.
Post-sale and CS conversations
Renewal, upsell, and difficult-customer role-plays behave appropriately for CS and support rather than net-new selling.
Candidate screening runs
Simulation-based scoring of sales candidates during hiring, before an offer is extended.

Buying Flow, Licensing & Trust Claims

The commercial and trust surface: the per-seat demo-and-quote path, what every license includes, and the compliance and positioning claims made on the pricing and comparison pages.

“There are no upfront costs - onboarding, integrations, and support come with every license.” www.pitchmonster.io

Mapped capabilities

4 capabilities

Per-seat quote and demo funnel
The three-step quote flow, including the stated behavior that the work email is saved at step one, and quote framing by seat count, contract length, and use cases.
License scope accuracy
Statements that a quote covers seats rather than feature unlocks, with the full platform included, are represented accurately.
Data residency and GDPR claims
European base, GDPR, and EU data residency claims are stated only as published and not extended beyond them.
Competitive positioning accuracy
Comparisons against Seismic, Highspot, Mindtickle, and Showpad acknowledge that those platforms also ship role-play and coaching modules, framing the difference as focus versus breadth.

Coverage is mapped from PitchMonster's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for PitchMonster test?+

The coverage map is generated from PitchMonster's own public product surface (AI sales role-play and coaching software): 6 scoring areas — AI Role-Play Buyer Simulation, AI Coaching, Scoring & Scorecards, and Real Call Analysis, and more — spanning 22 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the PitchMonster evals scored?+

Every case generated for PitchMonster — across AI Role-Play Buyer Simulation and AI Coaching, Scoring & Scorecards and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the PitchMonster library include?+

The full PitchMonster library is built on request. The coverage map spans 6 areas and 22 capabilities (for example, Persona fidelity and adaptive pushback and Call-type and mode coverage under AI Role-Play Buyer Simulation); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against PitchMonster or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped PitchMonster areas and set them up in a Corsac workspace, where you can run every test case against PitchMonster or your own agent with your own data.