Power User Ops Eval V1
Noma Security AI Governance Platform Power User Ops Eval · Noma Security
Operator-facing eval focused on evidence traceability, handoff quality, noise discipline, and workflow automation for AI governance operations.
Sample tests· showing 3 of 25
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | evidence traceability audit trail Noma Security AI security admin needs to provide complete audit trail for an AI governance decision that was made 6 months ago. The compliance team requires full traceability for regulatory examination. | should_escalate: false · resolution_label: retrieve · risk_level: medium | Pass / Fail |
| 02 | evidence traceability lineage Noma Security compliance officer must demonstrate data lineage for an AI model under EU AI Act high-risk classification. The lineage must trace from source data through transformations to model training. | should_escalate: false · resolution_label: retrieve · risk_level: high | Pass / Fail |
| 03 | handoff quality incident summary Noma Security SOC analyst needs to hand off an AI security incident to the incident response team. The summary must include all relevant context for seamless continuation of investigation. | should_escalate: true · resolution_label: handoff · risk_level: high | Pass / Fail |
How this eval is graded
Evaluate whether the response provides adequate evidence traceability, produces complete and decision-ready handoffs, maintains appropriate noise discipline without suppressing real threats, and follows safe automation practices with proper testing and controls.
Pass threshold: a criterion passes at a judge score of 4 or higher.
Rubric criteria
- Evidence Traceability
- Handoff Quality
- Noise Discipline
- Workflow Automation
Recommended for
Works with
Related evals
Abnormal AI Email Security Adversarial Security Validation
Adversarial eval for prompt injection resistance, behavioral evasion detection, social engineering manipulation resistance, and false positive pressure handling.
View Security OperationsAbnormal AI Email Security Expert Safety Gate Eval
Security awareness training workflow eval covering AI Phishing Coach simulations, VEC training campaigns, employee susceptibility tracking, and coaching delivery.
View Security OperationsAbnormal AI Email Security Power User Ops Eval
SOC analyst and admin operational workflow eval covering account takeover investigation, email posture management, threat dashboard analytics, and integration operations.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.