Self Host Cloud And Embed
n8n (self-host + Cloud) · n8n
Workflow Automation — n8n
Evaluates n8n's Self-host, Cloud & Embed across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Workflow Automation eval coverage.
About n8n
n8n is an open-source workflow automation platform — visually composed workflows of 1000+ nodes including AI/LangChain nodes (AI Agent, vector stores, memory, tools), with triggers (webhook/schedule/poll/form/chat), credentials with encryption at rest, queue-mode execution (Redis-backed workers), self-host (Docker/Kubernetes) and n8n Cloud options, and source-control/embed for teams.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator runs `docker run n8nio/n8n` without mounting a volume. After a container recreate, all workflows and credentials disappear. | Mount `/home/node/.n8n` to a host volume (SQLite + encryption key file). For production, use Postgres external to the container so n8n is stateless. Document the volume + DB + N8N_ENCRYPTION_KEY as the three durable state surfaces. | Pass / FailAi Platformcritical |
| 02 | n8n runs behind nginx terminating TLS at https://n8n.acme.com. Webhook URLs shown in the UI start with http://localhost:5678. | Set N8N_HOST, N8N_PROTOCOL=https, N8N_PORT (or WEBHOOK_URL=https://n8n.acme.com) so the displayed webhook URL matches the public hostname. Without this, partners register localhost URLs and receive nothing. Verify with a test webhook. | Pass / FailAi Platformcritical |
| 03 | Helm chart deploys 3 main pods sharing a ReadWriteOnce PVC for the n8n data directory. Two pods crashloop with 'database is locked'. | Single-main deployment: 1 main pod + N worker pods, all pointing at an external Postgres + Redis. Do not run multiple main pods backed by SQLite or a shared PV. If multi-main is required, enable N8N_MULTI_MAIN_SETUP_ENABLED + Postgres. | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- N8n
- Ai Platform
- Self Host Cloud And Embed
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Self Host Cloud And Embed eval for n8n n8n (self-host + Cloud) test?+
Evaluates n8n's Self-host, Cloud & Embed across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Workflow Automation eval coverage.
How is the Self Host Cloud And Embed eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Self Host Cloud And Embed pack for n8n n8n (self-host + Cloud) contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Self Host Cloud And Embed pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.