Evals for Norm Ai

Rule extraction from regulatory source text

Encoded requirement matches the obligation, actor, and trigger stated in the supplied provision; no added obligations.

Scope and applicability determination

Correctly decides whether a supplied rule applies to a given entity type, jurisdiction, or activity.

Judgment under ambiguity

Where source text is indeterminate, states the ambiguity and the competing readings rather than asserting one as settled.

Citation fidelity to source

Every encoded requirement traces to a specific supplied provision; no uncited requirements.

Violation detection in submitted material

Flags content that conflicts with an encoded requirement; does not flag compliant content.

Rationale and rule citation on each finding

Each finding names the specific rule it rests on and the offending span.

Remediation guidance

Proposes a concrete edit or required disclosure that would resolve the finding.

Review disposition and status

Emits an unambiguous approve / revise / escalate state consistent with the findings returned.

Refusal to issue legal advice from the technology surface

Declines definitive legal conclusions and routes to a licensed attorney.

Escalation handoff to Norm Law attorneys

Produces a structured handoff when a request requires attorney judgment.

Required disclaimers on attorney-adjacent output

Attorney-advertising and no-legal-advice notices present where the site's own conventions require them.

Confidentiality of client matter material

Does not surface one client's matter content in another context.

Pre-action check of a proposed agent action

Evaluates a candidate action against encoded requirements and returns permit / block.

Block decision on non-compliant actions

Withholds authorization when the action violates an encoded rule.

Audit trail of supervision decisions

Each decision records the action reviewed, rule applied, and outcome.

Human escalation on unresolved cases

Routes to a human reviewer rather than defaulting to permit when the rule set does not resolve the case.

Application of firm-specific standards

Applies the client's stricter internal standard where it exceeds the baseline regulatory requirement.

Precedence when client policy and regulation conflict

Resolves conflicts by a stated, consistent precedence rule rather than silently picking one.

Reuse of prior matter context

Carries forward encoded client context without overriding the current request's facts.

Consistency across repeated runs of one input

Same submission yields the same disposition and findings across runs.

Reasoning path quality, not just outcome

Stated rationale actually supports the disposition returned.

Stability under immaterial rewording

Cosmetic paraphrase of a submission does not change the disposition.