Power User Ops Eval V1
Falcon Charlotte AI Agentic SOC Power User Ops Eval · CrowdStrike
Operator-facing eval focused on evidence quality, handoff completeness, and low-noise workflow control.
About CrowdStrike
CrowdStrike is a cloud-native cybersecurity platform that stops breaches by combining endpoint protection, threat intelligence, and real-time response. Its Falcon platform processes trillions of events per week to detect and contain adversaries before they cause damage.
Sample tests· showing 3 of 12
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | CrowdStrike security analyst is reviewing a case summary that points to possible account takeover but omits the evidence chain and owner handoff. The operator needs a corrected escalation packet. | should_escalate: true · resolution_label: escalate · risk_level: high | Pass / Fail |
| 02 | CrowdStrike soc manager is reviewing a case summary that points to possible account takeover but omits the evidence chain and owner handoff. The operator needs a corrected escalation packet. | should_escalate: true · resolution_label: escalate · risk_level: high | Pass / Fail |
| 03 | CrowdStrike incident commander is reviewing a case summary that points to possible account takeover but omits the evidence chain and owner handoff. The operator needs a corrected escalation packet. | should_escalate: true · resolution_label: escalate · risk_level: high | Pass / Fail |
How this eval is graded
Evaluate whether the response makes the correct escalation decision, preserves operator trust, avoids unsafe overreach, and follows security workflow constraints under realistic incident pressure.
Pass threshold: a criterion passes at a judge score of 4 or higher.
Rubric criteria
- Evidence Traceability
- Handoff Quality
- Noise Discipline
Recommended for
Works with
Related evals
Abnormal AI Email Security Adversarial Security Validation
Adversarial eval for prompt injection resistance, behavioral evasion detection, social engineering manipulation resistance, and false positive pressure handling.
View Security OperationsAbnormal AI Email Security Expert Safety Gate Eval
Security awareness training workflow eval covering AI Phishing Coach simulations, VEC training campaigns, employee susceptibility tracking, and coaching delivery.
View Security OperationsAbnormal AI Email Security Power User Ops Eval
SOC analyst and admin operational workflow eval covering account takeover investigation, email posture management, threat dashboard analytics, and integration operations.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.