Eval directory

Evals for Haptik

Eval coverage for Haptik, mapped from its public product surface.

About Haptik

Jio Haptik is an enterprise platform for building conversational AI agents that handle customer support, sales, bookings, and lead qualification across chat and voice channels. It includes an agent builder with selectable underlying LLMs, voice agents with multilingual support, and Smart Agent Chat for handing off complex queries to human agents with AI co-pilot assistance. An analytics product tracks agent performance, sentiment, and SOP compliance through unified real-time dashboards.

Industry

enterprise conversational AI agent platform for customer experience

Website

www.haptik.ai

Use the eval library for Haptik

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Haptik?

6 scoring areas · 23 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

AI Agent Builder & Model Selection

The Agent Studio surface where teams configure agents without pre-built journeys and choose among leading underlying LLMs (GPT, Llama, Claude). Covers whether agent behavior stays consistent and configuration intent is honored as models are swapped.

“Experiment with leading AI models like GPT, llama, and Claude to find the best fit” www.haptik.ai

Mapped capabilities

4 capabilities

Model selection and swap behavior
Agent responses remain aligned to configured instructions when the underlying LLM is changed between supported providers.
Journey-free agent configuration
Agents handle queries without a pre-authored flow, per the support agent's stated 'no pre-built journeys' positioning.
Instruction and scope adherence
Agent stays within its configured use case and declines or redirects out-of-scope requests.
Use-case agent differentiation
Support, sales, booking, and lead qualification agents exhibit their distinct stated objectives rather than converging on generic assistance.

Voice AI Agents

Phone-channel automation covering intent detection, context-aware response, and call routing under real call conditions. The context claims advanced intent detection and routing based on actual need rather than rigid menu trees.

“context-aware responses, and multilingual fluency in 100+ languages” www.haptik.ai

Mapped capabilities

4 capabilities

Intent detection on spoken input
Correct intent extraction from free-form, unstructured caller speech rather than menu-style keyword input.
Dynamic call routing
Routing decisions reflect the caller's actual need; incorrect or premature routing is avoided.
Context retention across turns
Earlier caller-provided details persist and are reused later in the same call.
Noise and disfluency tolerance
Behavior under background noise, per the platform's own noise-cancellation positioning.

Illustrative example

Input: Caller says: "Hi, my flight tomorrow morning got moved and I need to change my hotel booking to match — I can't do it on the app."
Expected behavior: The agent identifies a booking modification intent tied to the hotel reservation and routes to the booking path, rather than defaulting to a generic support queue or reading out a menu of options.

Multilingual Conversation Quality

The claimed multilingual fluency across 100+ languages for voice and chat. Distinct from core voice handling because language coverage fails in its own characteristic ways — silent language drift, degraded intent accuracy, and mismatched response language.

Mapped capabilities

4 capabilities

Language detection and matching
Response language matches the customer's language without explicit selection.
Mid-conversation language switch
Agent follows when the customer changes language partway through.
Code-mixed input
Handling of mixed-language utterances common in the platform's core markets.
Intent accuracy parity across languages
Intent detection quality does not degrade materially outside English.

Human Handoff & Agent Co-Pilot

Smart Agent Chat: escalation of complex queries from AI to human agents in the omnichannel Agent Inbox, plus Co-Pilot assistance (chat summaries with sentiment, suggested responses, tone enhancer) and SLA management.

“Conversational AI agents that understand, interact, and enhance customer experiences—just like human agents would” www.haptik.ai

Mapped capabilities

4 capabilities

Escalation trigger correctness
Complex or sensitive queries hand off to a human; routine ones do not escalate unnecessarily.
Context transfer at handoff
The human agent receives an accurate summary; the customer is not asked to repeat information.
Co-Pilot response suggestions
Suggested replies are grounded in the actual conversation and prior resolutions, not fabricated.
Tone enhancer fidelity
Rephrasing adjusts tone while preserving the factual content and any commitments made.

Illustrative example

Input: Customer, after three turns establishing order #48812 was delivered damaged and a prior refund was denied, says: "This is unacceptable, I want to speak to a real person now."
Expected behavior: The agent hands off to a human rather than attempting further automated resolution, and the summary delivered to the human agent carries the order number, the damage claim, and the prior denied refund.

Omnichannel Continuity

The unified agent workspace and channel coverage named in the context — WhatsApp, voice, RCS, Instagram Direct, Facebook Messenger, and web — where a conversation may be picked up on any channel from a single interface.

Mapped capabilities

3 capabilities

Cross-channel conversation pickup
An agent resumes a conversation started on another channel with full history intact.
Channel-appropriate formatting
Responses respect the constraints and conventions of the delivery channel.
Identity continuity
The same customer is recognized across channels without duplicate threads.

Analytics & Insights Agent

The real-time analytics product covering KPI dashboards, sentiment measured across the full conversation arc, agent efficacy scoring, and SOP compliance tracking. Evaluated on whether derived metrics faithfully reflect the underlying conversations.

“Track agents’ performance with real-time analytics.” www.haptik.ai

Mapped capabilities

4 capabilities

Sentiment trajectory accuracy
Sentiment scored from conversation start to end reflects actual customer state, including recovery cases.
SOP compliance detection
Deviations from defined standard operating procedures are flagged, and compliant conversations are not falsely flagged.
Conversation summarization fidelity
Summaries capture the resolution and outstanding commitments without inventing details.
Query trend and gap surfacing
Recurring unhandled queries are identified as candidates for new conversational flows.

Coverage is mapped from Haptik's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Haptik test?+

The coverage map is generated from Haptik's own public product surface (enterprise conversational AI agent platform for customer experience): 6 scoring areas — AI Agent Builder & Model Selection, Voice AI Agents, and Multilingual Conversation Quality, and more — spanning 23 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Haptik evals scored?+

Every case generated for Haptik — across AI Agent Builder & Model Selection and Voice AI Agents and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Haptik library include?+

The full Haptik library is built on request. The coverage map spans 6 areas and 23 capabilities (for example, Model selection and swap behavior and Journey-free agent configuration under AI Agent Builder & Model Selection); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Haptik or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Haptik areas and set them up in a Corsac workspace, where you can run every test case against Haptik or your own agent with your own data.