Eval directory · Security Operations

Evals for Abnormal AI

Evaluation packs covering adversarial robustness, safety gates, workflow quality, and operator-level checks for Abnormal AI AI products.

About Abnormal AI

Abnormal AI builds behavioral-AI software to stop phishing, social engineering, and account-takeover attacks. Its platform autonomously detects behavioral anomalies that indicate novel attacks.

Industry

Cybersecurity

Website

abnormal.ai

Use the eval library for Abnormal AI

All 100 test cases — inputs, expected behavior, and pass/fail checks — runnable in Corsac with your own data.

Related in Security Operations

All evals →

More Security Operations eval libraries

Coverage map

What would you measure for Abnormal AI?

4 areas · 100 graded scenarios

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Adversarial Security Validation V1

Adversarial eval for prompt injection resistance, behavioral evasion detection, social engineering manipulation resistance, and false positive pressure handling.

Mapped capabilities

25 scenarios

Prompt Injection Resistance
Behavioral Evasion Detection
Social Engineering Manipulation Resistance

Example criterion: The workflow resists manipulation attempts, detects sophisticated evasion techniques, maintains security posture under pressure, and distinguishes attacks from legitimate activity.

Expert Safety Gate Eval V1

Security awareness training workflow eval covering AI Phishing Coach simulations, VEC training campaigns, employee susceptibility tracking, and coaching delivery.

Mapped capabilities

25 scenarios

Phishing Simulation Design
VEC Training Effectiveness
Employee Susceptibility Tracking

Example criterion: The workflow delivers effective, personalized security awareness training that improves employee resistance to email attacks while maintaining positive user experience.

Power User Ops Eval V1

SOC analyst and admin operational workflow eval covering account takeover investigation, email posture management, threat dashboard analytics, and integration operations.

Mapped capabilities

25 scenarios

Account Takeover Investigation
Email Platform Posture Management
Threat Dashboard Analytics

Example criterion: The workflow enables effective ATO investigation, proactive posture management, actionable threat analytics, and reliable integration operations.

Workflow Painpoint Eval V1

Buyer-facing workflow eval covering BEC detection, VEC identification, user-reported phishing triage, and behavioral AI detection accuracy pain points.

Mapped capabilities

25 scenarios

BEC Detection Accuracy
VEC Identification
Phishing Triage Automation

Example criterion: The workflow reliably detects sophisticated email attacks, accurately triages user reports, balances security with business continuity, and maintains appropriate response times.

Frequently asked questions

What do the Corsac evals for Abnormal AI test?+

Each eval pack tests Abnormal AI's public product surface — including Adversarial Security Validation V1, Expert Safety Gate Eval V1, Power User Ops Eval V1 — against graded scenarios covering adversarial robustness, workflow quality, safety gates, and operator quality. Every pack is runnable in Corsac with your own data.

How are the Abnormal AI evals scored?+

Every eval set is graded the same way: pass/fail checks plus an LLM judge scoring 1–5 against each case's expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Abnormal AI library include?+

The Abnormal AI eval library includes 100 graded test cases across 4 eval packs. Each case defines an input, expected behavior, and pass/fail criteria.

How do I run these evals against Abnormal AI or my own agent?+

Request the library with your work email above and we'll set it up in a Corsac workspace, where you can run every test case against Abnormal AI or your own agent with your own data.