Workflows And Nodes
n8n (self-host + Cloud) · n8n
Workflow Automation — n8n
n8n evals — Workflows & Nodes (relift v3 InfraRed)
About n8n
n8n is an open-source workflow automation platform — visually composed workflows of 1000+ nodes including AI/LangChain nodes (AI Agent, vector stores, memory, tools), with triggers (webhook/schedule/poll/form/chat), credentials with encryption at rest, queue-mode execution (Redis-backed workers), self-host (Docker/Kubernetes) and n8n Cloud options, and source-control/embed for teams.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | A Code node returns `return {json: {foo: 'bar'}}` instead of an array. Downstream Set node receives only one item even when upstream sent 50. | Per docs, every node must emit `items` as an array of {json, binary} entries; the Code node should return `return items` or `return [{json: ...}, ...]`. When a single object is returned, treat it as a one-item array — but operator should fix the node to preserve fan-out semantics. | Pass / FailAi Platformcritical |
| 02 | Operator writes `={{ $json.id }}` in a node that runs after a Merge node, expecting it to reference the original HTTP node's response. | `$json` always refers to the current input item, which after a Merge node is the merged item. To reach back to a specific upstream node, use `={{ $node['HTTP Request'].json.id }}` or `={{ $('HTTP Request').item.json.id }}`. Document the scope explicitly so the workflow survives refactors. | Pass / FailAi Platformhigh |
| 03 | A workflow imported from a 0.x export runs branches in a different order than the new `executionOrder: v1` default. Downstream Merge picks up data from the 'wrong' branch first. | Set `settings.executionOrder: 'v1'` on new workflows (n8n default) — branches run in declared connection order. Imported legacy workflows may use `'legacy'`; pin the imported setting explicitly so semantics are deterministic across upgrades. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- N8n
- Ai Platform
- Workflows And Nodes
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.