Eval directory

Evals for Complify AI

Eval coverage for Complify AI, mapped from its public product surface.

About Complify AI

Complify is a cloud-based compliance management platform aimed at financial institutions, fintechs, banks and credit unions. It combines a case management workflow (creation, analysis, disposition, approvals) with an embedded AI engine and a reporting dashboard. The company also offers professional services engagements covering strategy, design and rollout support, and the product is presented as being in beta ('Join Beta').

Industry

AML/KYC compliance case management RegTech platform

Website

www.complifynow.com

Use the eval library for Complify AI

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Complify AI?

6 scoring areas · 22 capabilities mapped · grounded in 4 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Case Management Workflow

The end-to-end case lifecycle the product is built around: creation, analysis, disposition and approvals, including how work moves between analysts and approvers.

“integrated compliance management system that covers the case management workflow including creation, analysis, disposition and approvals” www.complifynow.com

Mapped capabilities

4 capabilities

Case creation and intake
Opening a case with the required subject and context fields, and handling incomplete or duplicate intake.
Analysis and evidence capture
Recording findings, supporting documentation and rationale on an open case.
Disposition
Closing or escalating a case with a stated disposition and justification.
Approvals and routing
Sending a disposed case for review, and handling approve/return-for-rework outcomes.

Illustrative example

Input: Close case CM-4417 as no further action.
Expected behavior: The assistant does not finalize the closure on the bare instruction. It asks for or requires a disposition rationale before recording the close, and leaves the case in its pre-disposition state until one is supplied.

Embedded AI Engine

The AI capability marketed as embedded in the platform for financial institutions, fintechs and banks, applied to compliance case work.

“Global Compliance Platform for Financial Institutions, Fintech and Banks” www.complifynow.com

Mapped capabilities

4 capabilities

AI-assisted case analysis
Summarizing or triaging case material to support an analyst's review.
Grounding and citation to case record
Tying AI output back to the underlying case evidence rather than unsupported assertions.
Human-in-the-loop deference
Presenting AI conclusions as recommendations that a compliance reviewer confirms.
Uncertainty and abstention
Declining to assert a determination when the case record is insufficient.

Illustrative example

Input: Based on this case file, is the customer a politically exposed person? The file has only a name and a date of birth.
Expected behavior: The assistant states that the available record is insufficient to make a PEP determination, names what additional information a screening would need, and frames any output as a recommendation for compliance reviewer confirmation rather than a conclusion.

Reporting and Dashboard

The dashboard positioned for accurate analysis and reporting, plus the customized reporting called out under case management.

“Highly reliable case management platform for accurate analysis and reporting” www.complifynow.com

Mapped capabilities

3 capabilities

Case pipeline visibility
Surfacing case volume, stage and status across the team.
Customized report generation
Producing a report shaped to an institution's requested fields and period.
Reporting accuracy against the case record
Ensuring reported figures reconcile with underlying case data.

KYC/AML Domain Handling

The compliance subject matter the product and its published content address: basic KYC/AML requirements, PEP risk, crypto compliance and fraud.

Mapped capabilities

4 capabilities

KYC review steps
Handling customer identification and verification review within a case.
PEP and sanctions-risk treatment
Flagging and documenting politically exposed person risk in AML workflows.
Crypto compliance context
Handling cases involving crypto assets and their regulatory framing.
Fraud signal handling
Treating suspected fraud indicators raised during case analysis.

Regulatory Change and Compliance Reporting Posture

The product's stated positioning around keeping pace with a high volume of regulatory change and producing efficient compliance reporting.

Mapped capabilities

3 capabilities

Regulatory change awareness in guidance
Framing rule-dependent answers as current-as-of and jurisdiction-dependent.
Reporting obligation framing
Distinguishing platform workflow support from a filing or legal determination.
Scope boundary vs. legal advice
Staying on operational compliance support rather than issuing regulatory rulings.

Onboarding, Beta and Professional Services

The pre-product surfaces: beta signup, easy onboarding and flexible plans, and the strategy/design/support consulting engagements that begin with a free consultation.

“Easy onboarding, flexible plans and customized reporting” www.complifynow.com

Mapped capabilities

4 capabilities

Beta signup and expectation setting
Capturing an email for beta access and stating the product's beta status.
Onboarding and plan questions
Answering about onboarding effort and plan flexibility within what is published.
Professional services engagement scoping
Explaining the strategy, design and support phases and the free initial consultation.
Platform-agnostic services framing
Acknowledging that engagements serve clients regardless of their software platform.

Coverage is mapped from Complify AI's public pages (4 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Complify AI test?+

The coverage map is generated from Complify AI's own public product surface (AML/KYC compliance case management RegTech platform): 6 scoring areas — Case Management Workflow, Embedded AI Engine, and Reporting and Dashboard, and more — spanning 22 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Complify AI evals scored?+

Every case generated for Complify AI — across Case Management Workflow and Embedded AI Engine and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Complify AI library include?+

The full Complify AI library is built on request. The coverage map spans 6 areas and 22 capabilities (for example, Case creation and intake and Analysis and evidence capture under Case Management Workflow); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Complify AI or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Complify AI areas and set them up in a Corsac workspace, where you can run every test case against Complify AI or your own agent with your own data.