Eval directory

Evals for Attention

Eval coverage for Attention, mapped from its public product surface.

About Attention

Attention is an AI layer for go-to-market teams that ingests call recordings, CRM data, and emails to produce insights and automate sales workflows. It offers a natural-language query interface over past calls, automatic CRM field updates, and auto-drafted follow-up emails and Slack summaries. It also ships a directory of 100+ task-specific agents covering sales, revenue operations, enablement, marketing, and account management use cases.

Industry

AI sales agents / revenue intelligence platform

Website

www.attention.com

Use the eval library for Attention

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Attention?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Cross-Call Question Answering

Natural-language queries over past sales and customer calls, answered from transcripts, notes, and connected knowledge bases, with links back to the specific call moments that support the answer.

“Attention creates call notes than you can easily sync to your CRM with one click.” www.attention.com

Mapped capabilities

4 capabilities

Answer grounding in retrieved transcripts
Answers reflect what was actually said in indexed calls rather than plausible sales generalities.
Citation to specific call moments
Claims link to the call snippet or timestamp that supports them, per the documented linking behavior.
Cross-call trend synthesis
Win/loss reasons, objections, and product mentions aggregated across many calls rather than a single conversation.
Coverage limits and abstention
Behavior when the question falls outside indexed calls, notes, and connected knowledge bases.

Illustrative example

Input: Ask: "Which competitor did prospects bring up most in our Q3 discovery calls, and what did they say about pricing?"
Expected behavior: The answer names competitors and pricing concerns only from indexed Q3 discovery calls, links each claim to the specific call moment, and says so plainly if the indexed set does not cover the period.

GTM Data Synthesis and Reporting

The Super Agent layer over CRM, call recordings, emails, and connected systems (Salesforce, HubSpot, Snowflake, Slack, Notion, Google Sheets) that produces executive reporting and recurring summaries without migrations.

“Attention ingests all of your GTM data across CRMs, call recordings, emails and more” www.attention.com

Mapped capabilities

4 capabilities

Multi-source reconciliation
Combining CRM records with call and email evidence when sources disagree or one is stale.
Executive report construction
Roll-up reporting that states its own basis and scope of data covered.
Recurring scheduled summaries
Weekly Slack reports that stay accurate as underlying data changes between runs.
Drill-down on aggregate claims
Moving from a summary figure to the deals or calls behind it.

CRM Write-Back and Field Control

Automatic extraction of call insights into mapped CRM fields, including custom fields and objects, with append-versus-overwrite behavior and updates across the deal lifecycle.

“Attention supports custom fields and objects, so it fits right into your unique CRM setup” www.attention.com

Mapped capabilities

4 capabilities

Field mapping fidelity
Extracted content lands in the configured field and respects custom fields and objects.
Append versus overwrite semantics
Honoring the configured note-handling mode instead of silently replacing prior content.
Stage- and team-conditional updates
Updates tailored by deal stage or team as documented in the configuration.
Regional tagging and organization
Notes tagged and organized by region or role for global teams.

Illustrative example

Input: An opportunity note field is set to append mode and already contains a prior rep's summary. A new call ends and the agent writes its extracted notes to that field.
Expected behavior: The new call notes are added to the field while the prior summary remains intact and readable, and the update lands in the mapped field rather than a default notes field.

Follow-Up Drafting and Send Control

Post-call follow-up emails drafted from call insights and deal context, plus a Slack bot delivering daily rundowns and reminders, with automatic-send or review-first modes.

“Attention is the only AI sales tool that runs cross-call analyses for better coaching, deal insights, and more.” www.attention.com

Mapped capabilities

4 capabilities

Grounding of recap content
Takeaways and next steps in the draft trace to the conversation rather than invented commitments.
Review-first versus auto-send
Respecting the configured mode so nothing reaches a buyer without required review.
Tone and personalization to deal context
Messaging shaped by the account and conversation as described on the product page.
Daily rundown and reminder accuracy
Slack summaries and overdue-task reminders reflecting current call activity.

Agent Directory and Configuration

A directory of 100+ task-specific agents spanning sales, revenue operations, enablement, marketing, account management, operations, partnerships, product, and sales engineering, with search, category browsing, and access requests.

Mapped capabilities

4 capabilities

Agent selection for a stated task
Routing a described need to an appropriate directory agent rather than a superficially similar one.
Category and search navigation
Finding agents by the documented categories and search interface.
Scope boundaries per agent
An agent staying within its described job rather than expanding into adjacent workflows.
Access request handling
The request-access flow for agents not yet enabled for the account.

Risk Signals and Human-in-the-Loop

Agents that monitor conversations for deal risk, compliance red flags, and knowledge-base drift, with manager notification and configurable human verification before changes take effect.

“Flags red-flag phrases and critical risk signals in real time” www.attention.com

Mapped capabilities

4 capabilities

Deal risk and churn flagging
Surfacing red-flag language and risk signals with the evidence behind the flag.
Compliance red-flag detection
Alerting on privacy or regulatory mentions monitored during calls.
Verification gate before knowledge-base writes
Honoring human-in-the-loop configuration before product details are updated.
Escalation routing to managers
Notifying the right person when sentiment or risk signals warrant intervention.

Coverage is mapped from Attention's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Attention test?+

The coverage map is generated from Attention's own public product surface (AI sales agents / revenue intelligence platform): 6 scoring areas — Cross-Call Question Answering, GTM Data Synthesis and Reporting, and CRM Write-Back and Field Control, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Attention evals scored?+

Every case generated for Attention — across Cross-Call Question Answering and GTM Data Synthesis and Reporting and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Attention library include?+

The full Attention library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Answer grounding in retrieved transcripts and Citation to specific call moments under Cross-Call Question Answering); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Attention or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Attention areas and set them up in a Corsac workspace, where you can run every test case against Attention or your own agent with your own data.