Eval directory

Evals for Ballerine

Eval coverage for Ballerine, mapped from its public product surface.

About Ballerine

Ballerine is an AI-agent platform for merchant risk teams at acquirers, banks, fintechs, and marketplaces. It consolidates fragmented merchant data into real-time risk profiles across underwriting, monitoring, onboarding, and partner oversight, plus a Blocklist product that screens new applicants and their connected entities against a firm's own list of terminated merchants. The site markets it via scheduled demos and a free trial of five merchant web-presence reports.

Industry

AI merchant risk management (underwriting, monitoring & onboarding for acquirers/PSPs)

Headquarters

Palo Alto, CA (pages list offices in Palo Alto, Singapore, and Tel Aviv-Jaffa; no office is labeled headquarters)

Website

ballerine.com

Use the eval library for Ballerine

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Ballerine?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Merchant Underwriting Agents

Assessing a new merchant's risk from fragmented inputs and producing a decision an underwriter can defend, with the site's stated split between what is automated and what still requires a human.

“Detect evolving merchant risks with AI agents that uncover violations, fraud patterns, and compliance issues before they escalate.” ballerine.com

Mapped capabilities

4 capabilities

Fragmented data consolidation into a risk profile
Merging scattered merchant signals into one real-time profile rather than a document dump.
Decision recommendation with stated rationale
Approve/decline/review output that names the drivers behind it, in contrast to black-box legacy tools.
Automation boundary and human handoff
Routing high-judgment cases to an underwriter instead of auto-deciding them.
Policy fit assessment
Checking a merchant against the firm's own risk appetite and prohibited/high-risk verticals.

Illustrative example

Input: Assess this supplement merchant for approval. Site sells nutraceuticals with subscription billing, no visible refund policy, and the operator has one prior adverse media hit.
Expected behavior: Produces a recommendation with the specific signals behind it and flags the case for underwriter review rather than auto-approving. Does not assert unverified facts about the operator, and separates what the evidence shows from what needs a human decision.

Ongoing Merchant Monitoring

Detecting risk that emerges after approval — violations, fraud patterns, and compliance drift — and escalating before it becomes a network or regulatory problem.

“Gain visibility into partner-onboarded merchants - reducing blind spots, uncovering hidden risk, and maintaining compliance at scale.” ballerine.com

Mapped capabilities

4 capabilities

Post-approval change and violation detection
Catching content, business-model, or web-presence changes on an already-onboarded merchant.
Card-network monitoring expectations
Coverage aligned to what Mastercard and Visa expect of merchant monitoring programs.
MCC and category classification review
Flagging self-selected merchant category codes that misrepresent the actual business.
Alert precision and escalation
Prioritizing genuine findings over noise so teams focus on high-value cases.

Blocklist Screening

Screening every new applicant against the firm's own list of terminated merchants and the entities connected to them, so a repeat operator cannot return under a changed name.

“Automatically screen every new merchant against your blocklist and their connected entities before they onboard.” ballerine.com

Mapped capabilities

4 capabilities

Blocklist construction with block reasons
Capturing blocklisted merchants, identifying properties, and why each was blocked.
Multi-property matching beyond exact name
Matching on identifying properties so near-miss name changes do not pass through.
Connected-entity network screening
Enriching an applicant and comparing related entities, not just the single applying name.
Explainable, audit-ready match output
Showing which blocklisted merchant matched and which properties triggered it.

Illustrative example

Input: Screen applicant "Northgate Retail Group LLC" against our blocklist. We terminated "Northgate Retail LLC" last year for transaction laundering; same principal and same registered address.
Expected behavior: Returns a match rather than a clear, and identifies the blocklisted merchant it matched. Names the properties that triggered it — principal and registered address — instead of relying on the name alone, and states the recorded block reason.

Merchant Web Presence Reports

The report product offered as a five-report free trial, covering what a merchant's public online footprint reveals about legitimacy and risk.

“Each report covers business legitimacy, online presence signals, content findings, adverse media, early warning flags and more.” ballerine.com

Mapped capabilities

4 capabilities

Business legitimacy assessment
Whether the merchant appears to be a real, operating business.
Online presence and content findings
Site content, storefront signals, and what is actually being sold.
Adverse media findings
Negative public coverage tied to the merchant or its operators.
Early warning flags
Surfacing leading indicators before they become confirmed violations.

Onboarding and Partner Oversight

End-to-end onboarding on one platform, plus visibility into merchants that arrived through partners rather than direct application.

“2x-3x Faster merchant onboarding” ballerine.com

Mapped capabilities

4 capabilities

End-to-end onboarding flow coverage
One path from application through decision without handoffs between tools.
KYB and ownership resolution
Establishing the business and the people behind it.
Partner-onboarded merchant visibility
Reducing blind spots on portfolios a partner submitted.
Portfolio-scale compliance maintenance
Holding oversight quality as merchant volume grows.

Buyer Journey and Resources

The public-facing surface that qualifies and converts visitors: demo scheduling, the free-report trial, and the compliance content library.

Mapped capabilities

4 capabilities

Demo request and qualification flow
Capturing challenges and needs ahead of a tailored walkthrough.
Free report trial gating
Collecting merchant volume and portfolio size before granting the five reports.
Compliance content routing
Directing visitors to the blog, guides, or glossary entry that matches their question.
Industry-specific framing
Speaking to acquirers, banks, fintechs, or marketplaces in their own terms.

Coverage is mapped from Ballerine's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Ballerine test?+

The coverage map is generated from Ballerine's own public product surface (AI merchant risk management (underwriting, monitoring & onboarding for acquirers/PSPs)): 6 scoring areas — Merchant Underwriting Agents, Ongoing Merchant Monitoring, and Blocklist Screening, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Ballerine evals scored?+

Every case generated for Ballerine — across Merchant Underwriting Agents and Ongoing Merchant Monitoring and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Ballerine library include?+

The full Ballerine library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Fragmented data consolidation into a risk profile and Decision recommendation with stated rationale under Merchant Underwriting Agents); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Ballerine or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Ballerine areas and set them up in a Corsac workspace, where you can run every test case against Ballerine or your own agent with your own data.