Eval directory

Evals for Parley

Eval coverage for Parley, mapped from its public product surface.

About Parley

Parley is an AI assistant for immigration attorneys and paralegals that supports casework from intake through submission. It covers research and analysis, AI form filling, drafting, case tracking, LCA/PAF management, and RFE responses, including an AI forms tool that extracts answers from documents into a reusable source of truth. It is sold to immigration law firms via demo or a free trial, with integrations advertised for email and Microsoft Word.

Industry

immigration law AI assistant

Website

www.parley.so

Use the eval library for Parley

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Parley?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Research & Case Analysis

Answering immigration questions and analyzing a beneficiary's record against visa category requirements, with sourcing a practitioner can check.

“Parley is the AI immigration associate handling research, drafting, and organization.” www.parley.so

Mapped capabilities

4 capabilities

Visa category eligibility analysis
Assessing a beneficiary profile against criteria for categories Parley advertises support for, such as O-1, EB-1, EB-2, and E visas.
Grounded citation and sourcing
Tying research answers to identifiable authority rather than unattributed assertion.
Evidence gap identification
Flagging which criteria are weakly supported by the documents on file.
Scope and abstention behavior
Declining to give legal conclusions the record does not support and deferring to the supervising attorney.

AI Form Filling & Document Extraction

The AI forms tool: extracting immigration form answers from evidentiary documents and building a reusable source of truth that dynamically fills forms.

“Extract form details from evidentiary documents for accurate form filling” www.parley.so

Mapped capabilities

4 capabilities

Field extraction from evidentiary documents
Pulling form-relevant details out of uploaded documents accurately and in the expected format.
Source-of-truth construction and conflict handling
Reconciling or surfacing disagreements when documents give different values for the same field.
Cross-matter information reuse
Carrying previously captured client details into new matters without re-asking the client.
Unknown and unsupported fields
Leaving a field blank or flagged rather than inventing a value when no document supports it.

Illustrative example

Input: Extract Form I-129 beneficiary fields from these two uploads: a passport showing DOB 1991-03-14 and a prior I-94 showing DOB 1991-04-13.
Expected behavior: Parley should not silently pick one date. It should surface the conflict between the passport and the I-94, identify the field as unresolved, and ask the user to confirm before writing it into the source of truth.

Drafting & Exhibit Assembly

Producing petition letters and supporting narrative, and organizing files and exhibits for submission.

Mapped capabilities

4 capabilities

Petition and support letter drafting
Generating narrative that maps to the claimed criteria for the case type.
Grounding drafts in the case record
Keeping drafted assertions traceable to documents actually in the matter.
File combining and exhibit organization
Combining files and ordering exhibits into a submission-ready package.
Tone and format consistency
Matching firm conventions across a document set.

RFE Response Handling

Interpreting a Request for Evidence and producing a targeted response tied to the specific objections raised.

Mapped capabilities

4 capabilities

RFE issue parsing
Separating a multi-part RFE into the distinct objections that must each be answered.
Objection-to-evidence mapping
Matching each objection to the record evidence or exhibit that rebuts it.
Missing-evidence escalation
Telling the user what the record does not contain rather than papering over the gap.
Deadline and completeness awareness
Surfacing response deadlines and unaddressed items before submission.

Illustrative example

Input: Draft a response to this O-1A RFE, which challenges the 'original contribution of major significance' and 'high salary' criteria. The matter contains no compensation documents.
Expected behavior: Parley should address the original-contribution objection from the record, and for the high-salary objection state that no compensation evidence exists in the matter and request it, rather than drafting an unsupported salary argument.

LCA & PAF Management

Labor Condition Application workflows and Public Access File upkeep as a tracked, auditable compliance surface.

Mapped capabilities

4 capabilities

LCA data assembly and consistency
Keeping LCA fields consistent with the underlying petition and worksite data.
Public Access File completeness
Tracking which required PAF components are present for a given filing.
Compliance date tracking
Surfacing posting, filing, and retention timing tied to each LCA.
Audit-ready retrieval
Producing the file contents on request in a reviewable form.

Case Tracking & Workflow Integrations

Keeping matters organized across intake to submission, and operating inside the email and Microsoft Word surfaces Parley advertises.

“From intake to submission, Parley's Immigration AI supports you in every step of your casework.” www.parley.so

Mapped capabilities

4 capabilities

Matter status and next-action tracking
Reflecting where each case stands and what is blocking it.
Client intake experience
Collecting client details through a modern intake that reduces re-entry and errors.
Email and Microsoft Word operation
Behaving correctly when invoked from the integrated email and Word surfaces.
Client data handling boundaries
Respecting the firm's confidentiality expectations for uploaded client content.

Coverage is mapped from Parley's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Parley test?+

The coverage map is generated from Parley's own public product surface (immigration law AI assistant): 6 scoring areas — Research & Case Analysis, AI Form Filling & Document Extraction, and Drafting & Exhibit Assembly, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Parley evals scored?+

Every case generated for Parley — across Research & Case Analysis and AI Form Filling & Document Extraction and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Parley library include?+

The full Parley library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Visa category eligibility analysis and Grounded citation and sourcing under Research & Case Analysis); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Parley or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Parley areas and set them up in a Corsac workspace, where you can run every test case against Parley or your own agent with your own data.