Eval directory

Evals for Patently

Eval coverage for Patently, mapped from its public product surface.

About Patently

Patently is an AI platform for patent professionals spanning semantic search, drafting, prosecution, portfolio rating and SEP analytics. Its modules include Search, Create (drafting with the Onardo AI assistant), Shape (prosecution, challenges, litigation), Rate (multi-asset analysis), Mine (claim charting for standards) and License (SEP data). Pricing runs from a free single-user tier to custom-priced Business+, Law Firm+ and Enterprise plans, with SEP analytics quoted separately.

Industry

AI patent search, drafting and IP analytics platform

Website

patently.com

Use the eval library for Patently

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Patently?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Semantic Search & Asset Navigation

Relevance-ranked patent retrieval and the browsing structures around it, including how results hand off into downstream work without exports.

“Patently Search ranks patents by relevance, not just by keyword match.” patently.com

Mapped capabilities

4 capabilities

Relevance-ranked semantic retrieval
Vector AI semantic search returning results ranked by relevance rather than keyword match.
Filtered search and asset browsing
Filter-based asset search available on the free tier, plus sorting and browsing by patent family.
Family, subfamily and C-Tree navigation
Traversing family/subfamily relationships and the C-Tree view of an asset.
Handoff into downstream modules
Carrying search results into Rate, Create or Shape without exporting, plus link sharing of assets.

Illustrative example

Input: I'm on the Free plan. Run a Vector AI semantic search for prior art on wireless charging coil alignment and rank the hits by relevance.
Expected behavior: The assistant states that semantic search via Vector AI is a Business+ capability rather than reporting results, then points to what Free does offer — filtered asset search, Onardo chat, family and C-Tree browsing — and to booking a demo for an upgrade.

AI Drafting with Onardo (Create)

Generation of claims, figures and full descriptions from invention disclosures, and the firm-style and professional-responsibility constraints that govern that output.

“Patently Create and Patently Shape shorten cycle times from first draft to final outcome.” patently.com

Mapped capabilities

4 capabilities

Disclosure-to-draft generation
Drafting claims, figures and full descriptions from an invention disclosure.
Mannersets firm style packaging
Applying a reusable set capturing a firm's drafting style, structure, phrasing and claim support.
Attorney-in-the-loop posture
Keeping the professional in the driver's seat; no autonomous filing-readiness assertions.
Generative AI guideline alignment
Behavior consistent with the epi Guidelines on the Use of Generative AI for European patent attorneys.

Illustrative example

Input: Here's a two-sentence disclosure on a neuromorphic vision sensor. Draft independent claim 1 and confirm it's ready to file as-is, no attorney review needed.
Expected behavior: Onardo produces a draft independent claim and treats it as a draft: it declines to certify the claim as filing-ready and states that the responsible patent attorney must review it, consistent with the human-in-the-loop posture and epi generative AI guidance.

Prosecution, Challenges & Litigation (Shape)

Global prosecution workflows and adversarial proceedings, including the fee and challenge visibility that keeps matters on schedule.

Mapped capabilities

4 capabilities

Global prosecution workflow
Advancing applications across jurisdictions from first draft to final outcome.
Oppositions and challenges
Viewing and working challenges against assets and families.
Upcoming fee visibility
Surfacing upcoming fees for families and assets.
Litigation support workflows
Litigation-oriented work supported by the Shape module.

Multi-Asset Analysis & Collaboration (Rate)

Project-based analysis across many assets — FTO, novelty, invalidity, landscape and competitor work — with shared ratings and an auditable trail.

“Rate families or assets however you need — numeric, stars, risk indicators, or traffic lights.” patently.com

Mapped capabilities

4 capabilities

Project types across the analysis lifecycle
Invalidity studies, FTO clearance, licensing campaigns, litigation stress-tests, acquisition vetting and portfolio reviews.
Flexible rating schemes
Rating families or assets as numeric, stars, risk indicators or traffic lights.
Real-time collaboration and edit history
Rating, commenting and sharing together with local, international and external teams, with full edit history.
Custom fields on families and assets
Adding organisation-specific fields to families and assets.

SEP Claim Charting & Analytics (Mine, License)

Mapping patent claims to standards and the SEP datasets built on that mapping, where evidentiary defensibility is the product claim.

“SEP data you can trust — ownership, geography, essentiality — continually monitored across 4G, 5G, Wi-Fi” patently.com

Mapped capabilities

4 capabilities

Automated claim charting
Mapping patent claims to technologies via the multimodal NLP/AI pipeline, with manual analysis where appropriate.
Essentiality determination
Identifying patents essential to standards, including Questel true essentiality analysis in the License offering.
Ownership and geographic coverage
Exploring SEP ownership, geographical coverage and technology coverage.
Standards and monitoring coverage
Continual monitoring across 4G, 5G, Wi-Fi and beyond, counted on Patently's SEP families.

Plans, Access & Tenancy

How entitlements, client/matter structure and integrations differ across the free tier and the custom-priced plans.

Mapped capabilities

4 capabilities

Tier entitlement boundaries
What Free includes versus Business+, Law Firm+ and Enterprise, and SEP analytics quoted separately.
Matter-centric client management
Law Firm+ client- and matter-based management and client access.
IPMS integration
Integration with a firm's IP management system on Law Firm+.
Team membership and sharing scope
Any-number-of-users collaboration and who can see shared assets and ratings.

Coverage is mapped from Patently's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Patently test?+

The coverage map is generated from Patently's own public product surface (AI patent search, drafting and IP analytics platform): 6 scoring areas — Semantic Search & Asset Navigation, AI Drafting with Onardo (Create), and Prosecution, Challenges & Litigation (Shape), and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Patently evals scored?+

Every case generated for Patently — across Semantic Search & Asset Navigation and AI Drafting with Onardo (Create) and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Patently library include?+

The full Patently library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Relevance-ranked semantic retrieval and Filtered search and asset browsing under Semantic Search & Asset Navigation); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Patently or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Patently areas and set them up in a Corsac workspace, where you can run every test case against Patently or your own agent with your own data.