Eval directory

Evals for Version Story

Eval coverage for Version Story, mapped from its public product surface.

About Version Story

Version Story is a redlining platform for legal teams that compares document versions, generates redlines, and merges changes from multiple contributors. Its Compare product organizes draft history and produces instant visual redlines, while Analytics applies a rules-based redline engine to large document sets for diligence and doc review reports. It integrates with Claude and Slack, and runs on AWS with SOC 2 certified architecture and operational controls.

Industry

legal document redlining and contract comparison software

Website

versionstory.com

Use the eval library for Version Story

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Version Story?

6 scoring areas · 22 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Version Organization & Comparison

Ingesting drafts into a project and comparing versions, including the non-linear cases the Help Center calls out.

“Easily manage document versions, generate instant redlines, and confidently merge changes from multiple contributors.” www.versionstory.com

Mapped capabilities

4 capabilities

Document intake
Drag-and-drop onto the project canvas, Add button, and drag from Outlook; docx, doc, pdf.
Simple A-to-B comparison
Selecting an original and a modified version and producing a comparison.
Multi-version and non-sequential comparison
Comparing one draft against several versions, or against a version that is not its immediate predecessor.
Draft history navigation
Presenting the full version history of a document in an organized, navigable view.

Redline Generation & Output Fidelity

Producing the visual redline itself and the artifacts users take away from it.

Mapped capabilities

4 capabilities

Insertion and deletion marking
Struck and inserted text rendered at the clause and phrase level.
Renumbering and structural edits
Clause renumbering, moved sections, and heading changes reflected in the redline.
View modes
PDF, Track Changes, and Changed Pages views of the same comparison.
Export and download
PDF download and multi-file download behavior, including browser permission setup.

Merge & Conflict Handling

Combining edits from multiple contributors into one document while surfacing disagreement.

Mapped capabilities

3 capabilities

Multi-draft merge
Choosing versions, initiating the merge, and exporting the merged output.
Conflict detection
Flagging sections where multiple authors edited the same passage.
Author attribution
Preserving per-author tracked changes so Word can display them by color.

Illustrative example

Input: Merge two drafts of the same NDA in which two contributors each rewrote the Section 4.1 notice period differently.
Expected behavior: Section 4.1 is flagged as a merge conflict and the user is asked which version to keep. Neither edit is applied silently, and the merged output does not contain both rewrites concatenated into one clause.

Analytics & Diligence Reporting

Applying the rules-based redline engine across large document sets to produce diligence and doc review output.

“Our rules-based redline engine guarantees the precision you need to adopt LLM based technology.” www.versionstory.com

Mapped capabilities

4 capabilities

Bulk document processing
Running across large sets of .docx, .doc, and .pdf files.
Clause finding and match scoring
Per-clause findings with match percentages against a baseline.
Deviation triage
Separating items that need review from those ruled out at a match threshold.
Summary-to-source verification
Tying an AI summary line back to the underlying source redline span.

Illustrative example

Input: An Analytics run over 40 leases reports that termination requires sixty (60) days' written notice. Ask the product to show why that finding is correct.
Expected behavior: It returns the source document and the underlying redline span showing the thirty-to-sixty day change, along with the match score. It does not restate the summary as its own justification without pointing at a specific document.

Assistant & Workspace Integrations

Driving redlines from Claude and Slack rather than from the web app, per the documented connectors.

“Perfect redline precision directly within Claude.” www.versionstory.com

Mapped capabilities

3 capabilities

Claude connector setup
Installing the Version Story plugin and connector.
In-assistant redline requests
Locating a current and prior draft and returning a redline inside the assistant conversation.
Slack agent setup
Connecting a Slack agent to Version Story.

Security, Access & Account Controls

The protections and account terms the Security and Terms pages commit to.

“Role-based Access Control (RBAC) ensures users only have access to the information required for their role” www.versionstory.com

Mapped capabilities

4 capabilities

Encryption posture
TLS in transit, SSE-S3 (AES-256) at rest, AWS Certificate Manager certificates.
Role-based access control
Users see only what their role requires.
Audit logging
Actions recorded for traceability.
Account lifecycle
Registration accuracy obligations, user-initiated deletion, and company suspension or termination.

Coverage is mapped from Version Story's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Version Story test?+

The coverage map is generated from Version Story's own public product surface (legal document redlining and contract comparison software): 6 scoring areas — Version Organization & Comparison, Redline Generation & Output Fidelity, and Merge & Conflict Handling, and more — spanning 22 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Version Story evals scored?+

Every case generated for Version Story — across Version Organization & Comparison and Redline Generation & Output Fidelity and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Version Story library include?+

The full Version Story library is built on request. The coverage map spans 6 areas and 22 capabilities (for example, Document intake and Simple A-to-B comparison under Version Organization & Comparison); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Version Story or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Version Story areas and set them up in a Corsac workspace, where you can run every test case against Version Story or your own agent with your own data.