Workflows As Code And Api
n8n (self-host + Cloud) · n8n
Workflow Automation — n8n
n8n evals — Workflows-as-Code & API (relift v3 InfraRed)
About n8n
n8n is an open-source workflow automation platform — visually composed workflows of 1000+ nodes including AI/LangChain nodes (AI Agent, vector stores, memory, tools), with triggers (webhook/schedule/poll/form/chat), credentials with encryption at rest, queue-mode execution (Redis-backed workers), self-host (Docker/Kubernetes) and n8n Cloud options, and source-control/embed for teams.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator scripts a CI job that creates a workflow via POST /api/v1/workflows and supplies the API key in the Authorization header (Bearer). | Per docs, the public API authenticates via the X-N8N-API-KEY header, not Authorization: Bearer. Update the CI job to send X-N8N-API-KEY. Store the key in the CI secret store; rotate quarterly. Use a dedicated automation user. | Pass / FailAi Platformhigh |
| 02 | Operator scripts GET /api/v1/executions to fetch all executions for the last 30 days. The script reads only the first 100 and stops. | API responses paginate via `nextCursor`; loop until nextCursor is null / empty. Cap loop iterations defensively. Use server-side filters (workflowId, startedAfter) to narrow the scan rather than fetching everything. | Pass / FailAi Platformmedium |
| 03 | Deploy script PUTs new workflow JSON but doesn't re-activate. The new triggers are not registered until manual activation. | Replacing a workflow does not implicitly re-activate triggers. After PUT, call POST /workflows/{id}/activate (or deactivate-then-activate to force re-register). Verify the trigger is live with a synthetic test event before declaring deployment complete. | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- N8n
- Ai Platform
- Workflows As Code And Api
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.