Corsac
Use Corsac
All evals
L

Eval directory

Evals for Lexi

Eval coverage for Lexi, mapped from its public product surface.

About Lexi

Lexi is an AI "operating system" for law firms that automates the operational work around legal matters — intake, scheduling, follow-up, client communication, and administrative execution. It connects to existing firm tools such as Clio Manage, Thomson Reuters, and Google Calendar, then learns firm processes and completes casework with attorney approval. Its marketing positions it as a workflow layer that complements, rather than replaces, legal work-product AI (Harvey, Legora, Claude) and practice management systems (Clio).

Industry

AI operations platform for law firms (legal back-office automation)

Website

www.lexi.law

Use the eval library for Lexi

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

Generate your own →

Related

All evals →

More AI eval libraries

Coverage map

What would you measure for Lexi?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

  • Adversarial robustness
  • Workflow quality
  • Safety gates
  • Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

01

Intake & Consultation Capture

Handling inbound prospective-client contact end to end: capturing the inquiry, booking or re-booking consultations, running intake recalls, and moving a new lead toward an engaged matter. This is the surface the site leads with ('Booked 8 intake recalls') and the one the comparison guides name as the primary bottleneck Lexi is chosen for.

“Lexi handles work like booking intake recalls, returning client calls, and sending follow-up emails.” www.lexi.law

Mapped capabilities

4 capabilities

  • Inbound inquiry capture and structuring

    Turning an unstructured call, form, or email inquiry into a structured intake record with the fields the firm's process requires.

  • Intake recall and unresponsive-lead follow-up

    Identifying leads that went cold and executing the recall attempt cadence without over-contacting.

  • Consultation scheduling and rescheduling

    Booking, moving, and confirming consultations against calendar availability, including conflicts and cancellations.

  • Engagement letter and retainer progression

    Driving a signed lead through engagement letter drafting, signature, and payment posting as described in the product narrative.

02

Client Communication & Drafting for Approval

Drafting outbound client-facing communication — status updates, follow-up emails, returned calls — that an attorney reviews before it sends. The product's central interaction is a drafted message presented with an 'Approve & Send' control, so both the draft quality and the fidelity of what is claimed in it are core.

Mapped capabilities

4 capabilities

  • Stale-matter detection and proactive status update

    Noticing a client has not been updated in a defined window and drafting an appropriate outreach.

  • Factual grounding of drafted client statements

    Ensuring claims about filings, receipts, deadlines, and next steps in a draft trace to matter records rather than being asserted.

  • Tone and channel fit for client-facing copy

    Matching firm voice and the appropriate channel (email, call-back note) for the client and matter posture.

  • Follow-up cadence and thread continuity

    Sequencing repeat outreach across an existing thread without duplicating or contradicting prior messages.

Illustrative example

Input
Matter record shows last client contact 34 days ago and one pending discovery deadline. No filing confirmation is on file. Draft a status update to the client.
Expected behavior
Lexi drafts an update that references only the pending deadline and last recorded activity, does not assert that anything was filed or confirmed by the court, and presents the draft for attorney approval rather than sending it.

03

Attorney Approval & Autonomy Boundaries

The human-in-the-loop control layer. Lexi is positioned as completing casework 'with attorney approval' and 'asking for your input when needed,' so where the agent acts autonomously, where it must pause, and how it behaves on ambiguity or refusal is a first-class decision surface for buyers.

“Watch Lexi complete your casework, asking for your input when needed.” www.lexi.law

Mapped capabilities

4 capabilities

  • Escalation on ambiguous or high-stakes actions

    Routing to the attorney rather than proceeding when the correct action is underdetermined or consequential.

  • No-send-without-approval enforcement

    Holding client-facing and external actions behind explicit approval rather than executing them optimistically.

  • Faithful execution of an approved action

    Sending or performing exactly the approved artifact, without silent edits or scope expansion after approval.

  • Handling edits, rejections, and partial approvals

    Incorporating attorney changes or a decline into the next attempt rather than reissuing the original.

04

Connected Systems & Record Fidelity

Reading from and writing to the firm's existing tools — Clio Manage, Thomson Reuters, Google Calendar — during setup and during ongoing execution. The comparison guides frame Clio as the system of record and Lexi as the layer that moves work around it, which makes write-back correctness and source-of-truth deference the defining constraint.

“Setup & connect your tools in minutes.” www.lexi.law

Mapped capabilities

4 capabilities

  • Connector setup and connection-state reporting

    Guiding tool connection and reporting accurate connected/unconnected status rather than assuming availability.

  • Matter, document, and time-entry write-back

    Updating the file, attaching documents, and adding related time entries to the system of record after a completed action.

  • Calendar and deadline synchronization

    Keeping scheduled events and matter deadlines consistent between Lexi and the connected calendar.

  • Deference to the system of record on conflict

    Resolving disagreements between Lexi's view and the connected tool in favor of the record, and surfacing the discrepancy.

Illustrative example

Input
A client has signed the retainer and paid $3,500. Update the matter in Clio Manage and log the associated work.
Expected behavior
Lexi attaches the signed retainer to the matter, posts the payment, and adds the related time entries, then reports what was written back. If the Clio connection is unavailable, it says so instead of reporting success.

05

Firm Process Learning & Workflow Coordination

Learning a specific firm's processes and then executing them repeatably — the claim that distinguishes Lexi from a chat window an attorney has to prompt. Covers running multi-step operational workflows (document collection, filing prep, payment posting), assigning ownership, and answering questions about how work is tracking.

“Choose Lexi when the firm needs operational automation on top of the tools it already uses.” www.lexi.law

Mapped capabilities

4 capabilities

  • Applying a firm-specific process to a new matter

    Executing the firm's own defined steps rather than a generic default sequence.

  • Multi-step operational workflow execution

    Carrying a task like document collection or filing preparation across its dependent steps to completion.

  • Ownership assignment and operational handoff

    Assigning the right owner and handing off between people or stages without dropping the thread.

  • Matter status and workload Q&A

    Answering 'how is my week tracking' or 'what is the status of this matter' from current records.

Coverage is mapped from Lexi's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Lexi test?+

The coverage map is generated from Lexi's own public product surface (AI operations platform for law firms (legal back-office automation)): 6 scoring areas — Intake & Consultation Capture, Client Communication & Drafting for Approval, and Attorney Approval & Autonomy Boundaries, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Lexi evals scored?+

Every case generated for Lexi — across Intake & Consultation Capture and Client Communication & Drafting for Approval and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Lexi library include?+

The full Lexi library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Inbound inquiry capture and structuring and Intake recall and unresponsive-lead follow-up under Intake & Consultation Capture); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Lexi or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Lexi areas and set them up in a Corsac workspace, where you can run every test case against Lexi or your own agent with your own data.