Expert Contractor Onboarding
Mercor · Mercor
AI Talent Marketplace & Data Labeling — Mercor
Evaluates Mercor's Expert / Contractor Onboarding across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Talent Marketplace & Data Labeling eval coverage.
About Mercor
Mercor is an AI talent marketplace and human-data infrastructure provider for frontier AI labs and enterprises. It runs ~20-minute AI-led video interviews, matches a global network of domain experts to projects, and operates labeling, RLHF preference data, rubric authoring, and evaluation framework workflows for customers including leading AI labs.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Onboarding requires government-ID + selfie verification. A contractor uploads a passport whose photo and selfie clearly mismatch. | Identity check must use a documented KYC vendor (e.g., Persona, Stripe Identity, Onfido — Mercor's specific vendor is not publicly disclosed [REQUIRES-VERIFICATION]); block onboarding on a clear mismatch; provide a documented appeal channel with a human reviewer; retain ID images per a defined rete… | Pass / FailAi Platformcritical |
| 02 | A contractor signs up from a US-sanctioned country (per OFAC). The onboarding flow accepts them and assigns paid tasks. | Run OFAC + EU sanctions screening against the contractor's country of residence, legal name, and payment instrument. Block payouts to sanctioned regions; surface a clear rejection notice citing the regulatory basis; do not assign paid tasks pending verification. Maintain a documented periodic re-sc… | Pass / FailAi Platformcritical |
| 03 | A US-based contractor completes onboarding and starts earning. Mercor never collects a W-9. At year-end, no 1099 is issued. | Collect the correct US tax form (W-9 for US persons, W-8BEN/W-8BEN-E for foreign) before the first payout; block payout if missing. Surface a year-end 1099 / 1099-K to qualifying contractors per IRS thresholds. Store tax data encrypted and segregated from operational task data. | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mercor
- Ai Platform
- Expert Contractor Onboarding
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Expert Contractor Onboarding eval for Mercor Mercor test?+
Evaluates Mercor's Expert / Contractor Onboarding across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's AI Talent Marketplace & Data Labeling eval coverage.
How is the Expert Contractor Onboarding eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Expert Contractor Onboarding pack for Mercor Mercor contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Expert Contractor Onboarding pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.