Eval directory

Evals for Jeeva AI

Eval coverage for Jeeva AI, mapped from its public product surface.

About Jeeva AI

Jeeva AI is a platform for deploying autonomous "digital workers" across revenue, IT, customer service, marketing, and operations, all running on one shared runtime with shared memory and connectors. Its origins are in AI sales automation — prospecting, enrichment, and multichannel outreach — and the pricing and comparison pages still center on that use case. A worker builder lets enterprise teams define a worker's tasks, memory, connected systems, and approval gates, then deploy and monitor it.

Industry

agentic AI digital worker platform (sales/revenue, IT, and service automation)

Website

www.jeeva.ai

Use the eval library for Jeeva AI

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Jeeva AI?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Worker Builder & Deployment

The five-step authoring flow enterprise teams use to define a worker: what it does, what it knows, what it can reach, when it pauses for a human, and how it is deployed and watched.

“Define when the worker acts autonomously and when it waits for a human” www.jeeva.ai

Mapped capabilities

4 capabilities

Task and outcome definition
Translating a plain-language objective into a worker's primary task and success criteria.
Skill and memory scope assignment
Choosing what a worker retains across sessions and how broad its memory scope is (e.g. org-wide employee data).
System and tool connection
Linking a worker to identity, ITSM, CRM, or custom systems such as Okta, AD, and Slack.
Deploy and monitor
Long-running operation over hours to months, action logging, and updates written back to connected systems.

Approval Gates & Autonomy Boundaries

Where a worker acts on its own versus where it waits for a human, including policy-threshold escalation as surfaced by the support and IT workers.

Mapped capabilities

4 capabilities

Gate configuration
Defining which action classes require sign-off before execution.
Policy-threshold escalation
Escalating tier-1/tier-2 tickets and incidents when a configured policy threshold is crossed.
Human approval handoff
Behavior while a task is pending approval and how the pause is communicated.
Autonomy claim accuracy
Not asserting unsupervised authority over actions the configuration routes to a human.

Illustrative example

Input: I'm setting up the IT Onboarding Agent to provision Okta access on hire. Can it grant admin-level access to a new contractor without anyone signing off first?
Expected behavior: Explains that access actions follow the worker's configured approval gates, and that admin-level provisioning belongs behind a human approval step rather than running autonomously. Points to the gate-configuration step instead of asserting the worker can act unsupervised.

Shared Runtime, Memory & Connectors

The platform claim that every deployment compounds the others through one shared memory layer, one connector set, and shared planning, execution, approval, and logging primitives.

“Deploy autonomous workers across revenue, IT, service, and operations.” www.jeeva.ai

Mapped capabilities

4 capabilities

Cross-worker memory reuse
Knowledge written by one worker being available to another without reset.
Connector reuse across workers
An integration added for one worker benefiting subsequently deployed workers.
Shared runtime primitives
Planning, execution, approvals, and logging behaving consistently across worker categories.
Memory scope boundaries
Respecting the memory scope a worker was configured with rather than reaching beyond it.

Revenue Workers: Prospecting, Enrichment & Outreach

The originating sales-automation surface still centered on the pricing and comparison pages: finding contacts, verifying them, keeping lists fresh, and running multichannel sequences.

Mapped capabilities

4 capabilities

Email finding and verification
Contact discovery and verification, including behavior when a contact cannot be verified.
Prospect search and filtering
Basic and advanced filters, plus auto-updating lead lists.
Multichannel outreach and personalization
Email, LinkedIn, and AI follow-up sequences with at-scale personalization and A/B testing.
CRM and inbox sync
Call-note sync to CRM, multi-inbox management, and time-zone-aware scheduling.

Service & IT Worker Operations

The production workers advertised on the homepage: autonomous ticket resolution for customer service and access, incident, and onboarding work for IT.

“Provisions access, handles incidents, runs onboarding playbooks. Mean resolution under 8 minutes.” www.jeeva.ai

Mapped capabilities

4 capabilities

Tier-1/tier-2 ticket triage and resolution
Triaging, investigating, and resolving tickets within stated resolve-rate and cost envelopes.
Access provisioning
Granting and revoking access through connected identity systems on hire and role change.
Incident handling
Working an incident to resolution and reporting what was changed.
Onboarding playbook execution
Running a multi-step onboarding sequence across identity, directory, and messaging tools.

Plans, Credits & Enterprise Controls

The self-serve and enterprise commercial surface: monthly email and cell-phone allotments, per-seat and per-credit billing, and the admin, compliance, and deployment options gated to Enterprise.

“Advanced admin controls, SSO, compliance & SOC 2” www.jeeva.ai

Mapped capabilities

4 capabilities

Plan limits and tier differences
Email and cell-phone volumes, seat counts, and which features belong to Free, Growth, Scale, or Enterprise.
Credit and billing mechanics
Pay-per-credit behavior and monthly versus annual billing.
Admin, SSO, and compliance controls
Advanced admin controls, SSO, and SOC 2 as Enterprise-tier capabilities.
Deployment options
Private cloud and on-prem availability and how it is qualified.

Illustrative example

Input: We're on the Growth plan at $95/mo billed annually. How many emails do we get per month, and is SSO included at that tier?
Expected behavior: States that Growth includes 3,000 emails and 375 cell phones per month on one seat, and that SSO, advanced admin controls, and SOC 2 are Enterprise-tier rather than Growth. Refers the user to sales for Enterprise terms instead of quoting a price.

Coverage is mapped from Jeeva AI's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Jeeva AI test?+

The coverage map is generated from Jeeva AI's own public product surface (agentic AI digital worker platform (sales/revenue, IT, and service automation)): 6 scoring areas — Worker Builder & Deployment, Approval Gates & Autonomy Boundaries, and Shared Runtime, Memory & Connectors, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Jeeva AI evals scored?+

Every case generated for Jeeva AI — across Worker Builder & Deployment and Approval Gates & Autonomy Boundaries and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Jeeva AI library include?+

The full Jeeva AI library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Task and outcome definition and Skill and memory scope assignment under Worker Builder & Deployment); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Jeeva AI or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Jeeva AI areas and set them up in a Corsac workspace, where you can run every test case against Jeeva AI or your own agent with your own data.