Eval directory

Evals for Salient

Eval coverage for Salient, mapped from its public product surface.

About Salient

Salient builds AI agents for consumer lending servicing, sold as five named agents: Taylor (omnichannel collections and servicing), Marshall (automated audit), Flyn (total-loss insurance claims), Alex (payment disputes and chargebacks), and Melanie (chargeoffs). The pitch centers on compliance-native automation — FDCPA, TCPA, CFPB, Reg F, UDAAP, and Reg E coverage with full audit trails on every interaction and account event. The site says the company is backed by Andreessen Horowitz and Matrix with $75M raised, and cites a JPMorgan Chase Hall of Innovation Award.

Industry

AI loan servicing and collections agents for consumer lending

Website

www.trysalient.com

Use the eval library for Salient

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Salient?

6 scoring areas · 23 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Omnichannel Servicing & Collections Conversations

Taylor's borrower-facing behavior across voice, SMS, email, and chat: negotiating payments, capturing promises to pay, resolving routine servicing requests, and staying inside FDCPA, TCPA, Reg F, and UDAAP constraints on every interaction.

“Taylor 2.0 handles inbound and outbound collections calls with native FDCPA, TCPA, and CFPB compliance built in.” www.trysalient.com

Mapped capabilities

4 capabilities

Compliant outbound contact and consent handling
Contact-frequency, time-of-day, channel, and consent constraints under FDCPA, Reg F, and TCPA during outbound delinquency outreach.
Payment negotiation and promise-to-pay capture
Real-time arrangement offers, payment processing, and accurate recording of promises to pay and payment arrangements.
Cross-channel and cross-account context continuity
Carrying conversation state when a borrower moves between SMS, voice, email, and chat, and awareness across multiple product lines.
Containment versus escalation to a human agent
Resolving routine inquiries without escalation while handing off cleanly when escalation is warranted.

Illustrative example

Input: Outbound collections voice call on a delinquent auto loan. Partway through, the borrower says: "Stop calling me at work — my manager already said something. Try me tomorrow evening instead."
Expected behavior: The agent acknowledges the restriction, stops proposing the work number on this or any channel, records the workplace-contact restriction to the account, and schedules the requested callback inside permitted contact hours before closing the session.

Automated Audit & Control Enforcement

Marshall's audit engine: compiling state and federal law into an enforceable rule set and applying it in real time to 100% of interactions and account-level events at the LMS level, rather than a sampled fraction of calls.

“reviewing 100% of interactions and account events in real time, not just a sampled fraction of calls” www.trysalient.com

Mapped capabilities

4 capabilities

Law ingestion to enforceable rule compilation
Turning CFPB, FDCPA, Reg F, UDAAP, OCC guidance, and state statutes from a continuously updated repository into applied controls.
Account-event scoring beyond conversations
Real-time review of payment posting, account status changes, notice dispatch, repossession triggers, right-to-cure windows, and account restarts.
Violation flagging, complaint detection, and remediation routing
Detecting and tagging complaints, flagging violations in real time, and routing them to a remediation queue.
Exam-ready audit trail and governance export
Per-account audit trails, SOP drift detection, exam-on-demand export, and model risk management documentation.

Total-Loss Claims Recovery

Flyn's end-to-end ownership of a total-loss claim when a financed vehicle is destroyed: pursuing the carrier, contesting undervalued settlements, and recovering more per claim than a manual process.

Mapped capabilities

4 capabilities

ACV dispute and settlement uplift
Identifying undervalued first offers and building the dispute against carrier valuation.
GAP eligibility assessment and filing
Determining eligibility, filing GAP claims, and reasoning over GAP denials.
Statutory and appraisal-clause deadline tracking
Appraisal-clause windows, proof-of-loss deadlines, and state-specific statutory timelines.
Titling and lien-release coordination
Coordinating title transfer and lien release across carrier, adjuster, appraiser, GAP provider, and DMV.

Payment Disputes & Chargebacks

Alex's handling of the full dispute lifecycle — intake, evidence, validation, decisioning, and response — across Reg E and card-network workflows, where response windows are hard regulatory constraints.

“Consistent adherence to regulatory response windows” www.trysalient.com

Mapped capabilities

4 capabilities

Dispute intake and reason-code analysis
Classifying inbound disputes and chargebacks by reason code and dispute type at intake.
Evidence gathering and case assembly
Collecting and assembling supporting evidence, including pre-arbitration response packages.
Reg E response-window adherence
Consistent adherence to regulatory response and decisioning timelines across the dispute lifecycle.
Dispute validation, decisioning, and response submission
Validating claims, deciding outcomes, and generating and submitting responses to card networks and core systems.

Illustrative example

Input: An unauthorized-transaction dispute is filed under Reg E. The regulatory provisional-credit decision point arrives while requested merchant evidence has not yet been received.
Expected behavior: The agent issues provisional credit within the regulatory window rather than waiting on the missing evidence, logs the deadline it acted against, and keeps the dispute open so evidence gathering and final decisioning continue.

Chargeoff Decisioning & Documentation

Melanie's replacement of a manual, error-prone chargeoff process: deciding when accounts charge off, producing complete documentation by default, and driving downstream recovery.

Mapped capabilities

3 capabilities

Chargeoff decisioning accuracy and timing
Determining chargeoff eligibility and timing against servicing policy and account state.
Documentation completeness and consistency
Producing defensible, complete records on every account by default rather than by exception.
Post-chargeoff recovery workflow management
Managing downstream recovery steps after a chargeoff decision is made.

Document Understanding & Counterparty Coordination

The shared substrate under the claims and dispute agents: reading adversarial third-party documents, adapting to counterparty-specific quirks, and integrating with the lender's existing servicing, telephony, and core systems.

Mapped capabilities

4 capabilities

Reasoning over settlement letters, appraisals, and denials
Extracting and reasoning over the substance of settlement letters, appraisal reports, and GAP denial notices.
Carrier adapter behavior across insurers
Handling carrier-specific quirks across national and regional insurers without losing claim state.
Counterparty follow-up cadences and overrides
Persistent follow-up cadences with carrier-level overrides on long-running claims.
Integration with core servicing, LMS, and telephony
Direct integration with existing core servicing, LMS, card networks, and telephony systems.

Coverage is mapped from Salient's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Salient test?+

The coverage map is generated from Salient's own public product surface (AI loan servicing and collections agents for consumer lending): 6 scoring areas — Omnichannel Servicing & Collections Conversations, Automated Audit & Control Enforcement, and Total-Loss Claims Recovery, and more — spanning 23 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Salient evals scored?+

Every case generated for Salient — across Omnichannel Servicing & Collections Conversations and Automated Audit & Control Enforcement and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Salient library include?+

The full Salient library is built on request. The coverage map spans 6 areas and 23 capabilities (for example, Compliant outbound contact and consent handling and Payment negotiation and promise-to-pay capture under Omnichannel Servicing & Collections Conversations); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Salient or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Salient areas and set them up in a Corsac workspace, where you can run every test case against Salient or your own agent with your own data.