Evals for Cardamon

Relevance filtering against firm profile

Correctly keeps in-scope updates and suppresses ones that do not touch the firm's products, activities, or markets, with a stated reason.

Summarisation and categorisation

Faithful summaries of the source update and stable category assignment, without introducing claims absent from the source.

Urgency and business-impact assessment

Assigns urgency and impact tied to the firm's risk methodology, and links the change to affected business areas and products.

Recommended actions and findings reports

Produces actionable next steps and findings output that trace back to the specific update rather than generic compliance advice.

Article-level chunking and structuring

Dense legal text is decomposed into discrete, citable items that preserve article/section provenance.

Applicability determination for the firm

Surfaces items that apply given the firm's activities and context; does not over-apply obligations meant for other entity types.

Classification as contextual, guideline, or obligation

Distinguishes binding obligations from guidance and background context, consistently across a regulation.

Impact scoring and risk-tag mapping

Applies the customer's internal risk model and maps each obligation to their existing risk tags.

Policy and control ingestion at usable granularity

Internal documentation is chunked finely enough that a control statement can be matched to a specific obligation.

Coverage and missing-coverage identification

Correctly reports an obligation as covered, partially covered, or unaddressed, and does not claim coverage from loosely related text.

Remediation suggestions tailored to frameworks

Proposed remediations reference the firm's own frameworks and the specific gap, not boilerplate.

Traceability of match, miss, and suggestion

Each result cites the obligation and the internal document passage that drove it.

Answer grounding and source citation

Answers cite the mapped regulatory provisions they rest on and stay within the customer's corpus.

Abstention outside the mapped corpus

Declines or flags scope limits when a question concerns a jurisdiction or instrument not mapped, instead of answering from general knowledge.

Cross-jurisdiction and L1–L4 comparison

Compares requirements across jurisdictions and regulatory levels without conflating them.

Harmonisation of common versus unique obligations

Identifies overlapping requirements across frameworks and isolates the jurisdiction-specific remainder.

Firm profile applied to outputs

Products, activities, and markets in the profile visibly drive relevance, applicability, and impact decisions.

Custom risk methodology and risk tags

Uses the customer's own risk model and tag taxonomy rather than a default severity scheme.

Source configuration honoured

Respects which regulatory feeds and sources the customer has enabled or excluded.

Consistency of context across products

The same firm context yields non-contradictory conclusions between obligation mapping, gap analysis, and search.

Edits preserved with audit trail

Human edits to an assessment persist and are recorded alongside the original AI output.

Assignment and review progress

Items can be assigned to teammates and their review state is reflected accurately in dashboards.

Filtering on assessment points

Results can be filtered on any assessment attribute and the filtered set matches the underlying data.

Sharing with intact source links

Shared findings retain working links back to the originating regulatory source.