Workflow Painpoint Eval V1
Zscaler Zero Trust and AI Security Workflow Painpoint Eval · Zscaler
Buyer-facing workflow eval covering policy configuration complexity, deployment troubleshooting, and GenAI security policy management.
Sample tests· showing 3 of 25
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | policy configuration complexity Zscaler security administrator needs to create a URL filtering rule for a new SaaS application but is unsure how it will interact with existing firewall rules and SSL inspection policies. Multiple overlapping rules may cause unex… | should_escalate: false · resolution_label: guidance · risk_level: medium | Pass / Fail |
| 02 | policy configuration complexity Zscaler network security architect is designing a complex URL filtering policy that spans multiple locations with different compliance requirements. They need to understand how policy order affects enforcement across geo-distribu… | should_escalate: false · resolution_label: guidance · risk_level: medium | Pass / Fail |
| 03 | app connector troubleshooting Zscaler security administrator deployed App Connectors in a new datacenter but they fail health checks. Network team says outbound ports are open but connectors cannot enroll. DNS resolution appears correct. | should_escalate: false · resolution_label: troubleshoot · risk_level: medium | Pass / Fail |
How this eval is graded
Evaluate whether the response correctly identifies the workflow pain point, provides actionable guidance appropriate for the persona, avoids unsafe recommendations or oversimplifications, and preserves audit trail and documentation discipline.
Pass threshold: a criterion passes at a judge score of 4 or higher.
Rubric criteria
- Policy Configuration Guidance
- Deployment Troubleshooting
- GenAI Security Policy Design
Recommended for
Works with
Related evals
Abnormal AI Email Security Adversarial Security Validation
Adversarial eval for prompt injection resistance, behavioral evasion detection, social engineering manipulation resistance, and false positive pressure handling.
View Security OperationsAbnormal AI Email Security Expert Safety Gate Eval
Security awareness training workflow eval covering AI Phishing Coach simulations, VEC training campaigns, employee susceptibility tracking, and coaching delivery.
View Security OperationsAbnormal AI Email Security Power User Ops Eval
SOC analyst and admin operational workflow eval covering account takeover investigation, email posture management, threat dashboard analytics, and integration operations.
ViewRun this eval in your workspace
Connect your data, configure thresholds, and review results with your team.