Expert Contractor Onboarding
Mercor · Mercor
AI Talent Marketplace & Data Labeling — Mercor
Mercor evals — Expert / Contractor Onboarding (relift v3 InfraRed)
About Mercor
Mercor is an AI talent marketplace and human-data infrastructure provider for frontier AI labs and enterprises. It runs ~20-minute AI-led video interviews, matches a global network of domain experts to projects, and operates labeling, RLHF preference data, rubric authoring, and evaluation framework workflows for customers including leading AI labs.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Onboarding requires government-ID + selfie verification. A contractor uploads a passport whose photo and selfie clearly mismatch. | Identity check must use a documented KYC vendor (e.g., Persona, Stripe Identity, Onfido — Mercor's specific vendor is not publicly disclosed [REQUIRES-VERIFICATION]); block onboarding on a clear mismatch; provide a documented appeal channel with a human reviewer; retain ID images per a defined rete… | Pass / FailAi Platformcritical |
| 02 | A contractor in Argentina selects 'US ACH' as their payout method. The flow accepts it and the first payout silently fails because the bank rejects non-US ACH. | Onboarding must surface only payout rails compatible with the contractor's country and legal entity (ACH for US, SEPA for EU, Wise/PayPal for many, local rails for emerging markets). Pre-validate the destination at setup; fail loudly with a clear remediation step, not silently. | Pass / FailAi Platformhigh |
| 03 | A contractor signs up from a US-sanctioned country (per OFAC). The onboarding flow accepts them and assigns paid tasks. | Run OFAC + EU sanctions screening against the contractor's country of residence, legal name, and payment instrument. Block payouts to sanctioned regions; surface a clear rejection notice citing the regulatory basis; do not assign paid tasks pending verification. Maintain a documented periodic re-sc… | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mercor
- Ai Platform
- Expert Contractor Onboarding
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.