Evals for depthfirst

Broken authorization and IDOR detection

Identifying missing or incorrect access checks, object-level authorization gaps, and subscription tier bypasses by reasoning about intended behavior.

Privilege escalation and logic flaws

Surfacing escalation paths and business-logic defects that depend on application semantics rather than known-bad code patterns.

Cross-service data flow mapping

Tracing inputs in one service to vulnerable sinks in another to expose attack paths that exist only between components.

Attack chain composition

Connecting individually low-severity findings into an exploitable path through data flow and control logic.

Evidence trails and exploitation conditions

Stating the preconditions under which a finding is exploitable and the evidence supporting that conclusion.

Dynamic exploit confirmation

Running a dynamic test against the running application when stated conditions are met, to confirm rather than assert exploitability.

False-positive suppression

Withholding or downgrading findings that cannot be shown reachable or exploitable.

Vulnerability discovery model behavior

Behavior attributed to the dfs-large1 model for vulnerability discovery and validation.

Malicious package blocking

Blocking packages identified as malicious anywhere they are being pulled into the organization.

Novel malware detection latency

Detecting previously unseen malware in dependencies within the claimed time window.

Ecosystem coverage

Analysis across open-source ecosystems, including the Ruby gem work described in depthfirst Labs research.

Block decision explanation

Communicating why a package was blocked so a developer can act on the decision.

Pre-merge change review

Reviewing diffs for vulnerabilities, sensitive data exposure, and malware prior to merge.

Fix generation matching codebase conventions

Proposing concrete code changes that follow existing project conventions.

Applicable PR comments and one-click fixes

Delivering suggestions developers can apply directly in the pull request without context-switching.

AI-generated code coverage

Applying the same review path to changes authored by AI agents as to human-authored changes.

Context-based credential identification

Identifying credentials by their authentication role in code rather than by pattern or predefined format.

Live secret validation

Testing each discovered credential against the service it authenticates to and attaching validation status.

Hidden backdoor discovery

Finding code paths where specific inputs return authentication tokens or privileged access from within business logic.

PII discovery across codebase and data files

Surfacing personally identifiable information across code and ingested or API-connected data sources.

Event-to-action routing

Turning a security event into repeatable actions routed to the right tool, team, and process without a manual handoff.

Unified coverage across surfaces

Protecting code, dependencies, secrets, and runtime through one integrated system.

Shared context for humans and AI agents

Exposing the same platform context to human engineers and to AI agents operating on the codebase.

Model and provider portability

Preserving consistent enterprise outcomes across model, provider, policy, and access changes.