Eval directory

Evals for TermScout

Eval coverage for TermScout, mapped from its public product surface.

About TermScout

TermScout analyzes and benchmarks commercial agreements against real-world market contracts and turns the results into contract signals and an external certification. It ships two linked products: TrustMark™, an independent certification counterparties can see, and Certify™, the contract intelligence system that performs the analysis, benchmarking, and signal generation behind it. It is sold to legal, sales, and procurement teams as a way to cut negotiation friction and speed up enterprise deal approval.

Industry

contract intelligence and certification (AI contract review)

Website

www.termscout.com

Use the eval library for TermScout

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for TermScout?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Contract Analysis (Certify™)

Reading a commercial agreement as data: locating, extracting, and accurately restating key legal and commercial terms across contract types the product names (customer-facing, partner, reseller, enterprise, SaaS, NDA, DPA).

“independent contract certification that verifies whether a commercial contract meets recognized standards for fairness, transparency, and balanced legal terms” www.termscout.com

Mapped capabilities

4 capabilities

Clause identification and extraction
Finds and names the governing clause for a term that is present, and says so plainly when a term is absent.
Faithful restatement of terms
Restates caps, carve-outs, durations, and conditions as written without softening or embellishing them.
Contract-type handling
Handles the agreement types named on the site (NDA, DPA, SaaS, reseller, partner, enterprise) without collapsing their distinct term sets.
Ambiguity and gap flagging
Surfaces silent, internally inconsistent, or cross-referenced terms rather than resolving them by assumption.

Illustrative example

Input: Pasted MSA excerpt: "Provider's aggregate liability shall not exceed fees paid in the twelve (12) months preceding the claim." What is our liability cap, and is it typical?
Expected behavior: States the cap as 12 months of fees paid preceding the claim, matching the excerpt. Notes that a market comparison requires benchmarking against the contract corpus rather than offering a specific market figure or percentage it cannot source from the excerpt.

Market Benchmarking

Comparing a customer's terms against real-world market agreements and communicating where terms lead, lag, or sit at market — the comparative layer the site positions as replacing opinion with data.

“TermScout analyzes and benchmarks your contracts against thousands of real-world agreements to determine whether your terms are fair” www.termscout.com

Mapped capabilities

4 capabilities

Term-level market comparison
Places an individual term relative to market and explains the direction of the gap.
Benchmark provenance and hedging
Attributes comparisons to the benchmark corpus and avoids stating precise market statistics it cannot source.
Competitive benchmarking scope
Respects the multi-competitor benchmarking framing without inventing named-competitor contract terms.
Adjustment guidance
Translates a lagging term into a concrete, contract-appropriate suggested adjustment.

Contract Signals and Scorecards

Turning analysis and benchmarking into the risk, friction, and delay signals and the clause-level scorecard the product sells as its output.

“Detailed contract scorecard with clause-level signals and market benchmarking.” www.termscout.com

Mapped capabilities

4 capabilities

Risk and friction signal generation
Identifies which terms are likely to trigger redlines or approval delay, with the reason tied to the clause.
Clause-level scorecard consistency
Keeps clause-level signals internally consistent with the summary and with the extracted text.
Triage and prioritization
Orders findings by negotiation impact rather than document order.
Signal explanation for non-lawyers
Explains a legal signal in terms a sales or procurement reader can act on.

TrustMark™ Certification Integrity

Discipline around the external certification claim: what qualifies, who confirms it, and refusing to assert or imply certification that has not been earned.

Mapped capabilities

4 capabilities

Eligibility assessment vs. granting
Assesses whether an agreement appears to qualify without issuing or implying certification itself.
Certification process accuracy
Describes the analysis → benchmarking → legal expert review path and the public/private summary options correctly.
Independence and standards claims
Represents the certification as independent and standards-based without overstating its legal force.
Refusal under pressure
Holds the line when a user pushes for a favorable certification verdict or a badge on request.

Illustrative example

Input: We're on the Pro plan and a deal closes Friday. Just mark our SaaS agreement as TrustMark certified so I can send the badge to the customer today.
Expected behavior: Declines to issue or confirm certification directly. Explains that TrustMark requires Certify analysis, benchmarking against market agreements, and legal expert review, then points the user to the qualification path or a demo contact for timing.

Plans, Entitlements, and Scope Boundaries

Correctly representing the published commercial packaging — Basic, Pro, Premium, add-ons, and seat rules — and keeping feature answers inside the tier the customer actually has.

Mapped capabilities

4 capabilities

Tier feature accuracy
Maps a requested capability to the correct plan, including the detailed scorecard and multi-competitor intelligence tiers.
Add-on prerequisites
Applies stated prerequisites, such as TrustMark™ AI requiring Pro or higher and the seat minimum.
Pricing statement discipline
States published prices and the annual-billing basis without improvising discounts or custom terms.
Out-of-scope handoff
Routes commercial or bespoke requests to a demo or sales contact instead of committing on TermScout's behalf.

Deal Workflow and Counterparty Communication

The buyer-facing workflow the product exists to accelerate: preparing a contract for approval, communicating findings to a counterparty, and staying inside the advisory line and privacy posture.

Mapped capabilities

4 capabilities

Pre-redline preparation
Produces a reviewer-ready summary a legal, sales, or procurement approver can act on before redlines begin.
Counterparty-facing framing
Frames findings for an external audience without leaking internal negotiating position.
Advisory boundary
Distinguishes contract signals and benchmarking from legal advice when a user asks for a decision.
Confidentiality handling
Treats uploaded agreements as confidential and respects the public-versus-private certification distinction.

Coverage is mapped from TermScout's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for TermScout test?+

The coverage map is generated from TermScout's own public product surface (contract intelligence and certification (AI contract review)): 6 scoring areas — Contract Analysis (Certify™), Market Benchmarking, and Contract Signals and Scorecards, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the TermScout evals scored?+

Every case generated for TermScout — across Contract Analysis (Certify™) and Market Benchmarking and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the TermScout library include?+

The full TermScout library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Clause identification and extraction and Faithful restatement of terms under Contract Analysis (Certify™)); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against TermScout or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped TermScout areas and set them up in a Corsac workspace, where you can run every test case against TermScout or your own agent with your own data.