Eval directory

Evals for Warmly

Eval coverage for Warmly, mapped from its public product surface.

About Warmly

Warmly is an AI-native, human-supervised platform of autonomous revenue agents for B2B go-to-market teams. It de-anonymizes website visitors at the person level and engages them through AI chat, email, LinkedIn/Meta ads, popups, and personalized microsites, with a TAM Agent for outbound and an Inbound Agent for site traffic, unified by a Context Graph. It is sold in tiered annual/quarterly plans with credit allowances and integrates with CRM, marketing automation, advertising, and email tools; the company has announced an agreement to be acquired by HubSpot.

Industry

AI go-to-market (GTM) revenue agents / website visitor de-anonymization

Website

www.warmly.ai

Use the eval library for Warmly

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Warmly?

6 scoring areas · 23 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Person-Level Visitor De-anonymization

Identifying website visitors as individual people and companies, filtering them to ICP, and getting them to the right rep quickly.

“Warmly’s AI de-anonymizes website visitors at the person level” www.warmly.ai

Mapped capabilities

4 capabilities

Person and company identification
Explaining what the pixel resolves at the person level vs. company level for site traffic.
ICP filtering
Narrowing identified traffic to accounts and contacts that match the customer's ideal profile.
Real-time and Slack alerts
Notifying the right team member when a qualifying visitor is on the site.
Lead routing
Assigning identified visitors to owners and downstream follow-up paths.

Illustrative example

Input: Does your pixel identify individual people or only companies, and can identified visitors sync into Salesforce?
Expected behavior: Confirms identification at the person level in addition to companies, and confirms Salesforce as a supported CRM sync destination without claiming support for a CRM outside the listed integrations.

Inbound Agent Engagement

Meeting site visitors live and converting them through chat, offers, popups, microsites, and follow-up.

“Engages them across chat, email, LinkedIn ads, Meta ads” www.warmly.ai

Mapped capabilities

4 capabilities

AI chat with Context Graph context
Conversing with a visitor using the account and interaction context the Graph supplies.
Warm calling and live video chat
Initiating a live conversation with a visitor at the moment of observed interest.
Intent-based popups and microsites
Triggering smart popups and personalized microsites based on visitor intent.
Automated chat and email follow-up
Generating and sending follow-up after a chat session, including AI-written email.

TAM Agent Outbound

Going to buyers before they arrive: segmenting the addressable market and running coordinated outbound.

Mapped capabilities

4 capabilities

TAM breakdown and account scoring
Decomposing the total addressable market into scored, prioritized accounts.
Buying committee mapping
Automatically identifying the relevant roles within a target account.
Cross-channel signal tracking
Tracking buying signals across the channels Warmly monitors.
Coordinated email and ads
Running email alongside LinkedIn and Meta ad touches against the same accounts.

Context Graph and Account Intelligence

The unified record of site visits, emails, and meetings that agents consult before they act.

Mapped capabilities

3 capabilities

Unified interaction history
Consolidating every site visit, email, and meeting into a single account view.
Pre-action context retrieval
Ensuring an agent reads available account context before engaging.
Account activity timeline
Surfacing everything that has happened to an account in reviewable order.

Integrations and Access Surfaces

Connecting Warmly to the surrounding GTM stack and reaching it through app, API, or MCP.

Mapped capabilities

4 capabilities

CRM sync
Syncing identified visitors and activity with HubSpot, Salesforce, and other supported CRMs.
Marketing automation and advertising
Connecting to MAP tools and to LinkedIn, Google, and Meta advertising destinations.
Email, SEP, and webhooks
Handing off to sequencing and email tools or to custom webhook endpoints.
App, API, and MCP access
Reaching the same capabilities through the listed access surfaces.

Plans, Credits, and Commercial Policy

Tiered plans, credit allowances, feature gating, and the human-supervised, GDPR-compliant posture.

“4.8 / 5 based on 200+ reviews | GDPR Compliant” www.warmly.ai

Mapped capabilities

4 capabilities

Tier and feature gating
Stating which capabilities belong to De-anonymization, Inbound Chat, or Inbound Autopilot.
Annual vs. quarterly pricing
Quoting the correct listed price and savings framing for each billing term.
Credit allowances
Explaining the monthly credit floor included with plans.
Human supervision and GDPR posture
Representing the human-supervised model and GDPR-compliant claim accurately.

Illustrative example

Input: What does AI Inbound Autopilot cost if we pay quarterly instead of annually, and how many credits do we get?
Expected behavior: States $9,750 per quarter for AI Inbound Autopilot, notes it includes from 10K credits per month, and directs the buyer to sales rather than implying self-serve signup or inventing a discount.

Coverage is mapped from Warmly's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Warmly test?+

The coverage map is generated from Warmly's own public product surface (AI go-to-market (GTM) revenue agents / website visitor de-anonymization): 6 scoring areas — Person-Level Visitor De-anonymization, Inbound Agent Engagement, and TAM Agent Outbound, and more — spanning 23 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Warmly evals scored?+

Every case generated for Warmly — across Person-Level Visitor De-anonymization and Inbound Agent Engagement and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Warmly library include?+

The full Warmly library is built on request. The coverage map spans 6 areas and 23 capabilities (for example, Person and company identification and ICP filtering under Person-Level Visitor De-anonymization); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Warmly or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Warmly areas and set them up in a Corsac workspace, where you can run every test case against Warmly or your own agent with your own data.