Corsac
Use Corsac
All evals
Merge

Eval directory

Evals for Merge

Eval coverage for Merge, mapped from its public product surface.

About Merge

Merge is an integrations platform that lets companies build once and connect to many third-party APIs, tools, and LLMs without maintaining the integration stack themselves. Its pages describe products spanning Unified API, Agent Handler, and a Gateway/Embedded Routing Stack for giving customers their own model controls. It targets product and engineering teams shipping large numbers of integrations, with emphasis on security and GDPR-compliant data handling.

Industry

unified API / integration infrastructure for products and AI agents

Website

www.merge.dev

Use the eval library for Merge

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

Generate your own →

Related

All evals →

More AI eval libraries

Coverage map

What would you measure for Merge?

6 scoring areas · 22 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

  • Adversarial robustness
  • Workflow quality
  • Safety gates
  • Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

01

Unified API integration surface

Building once against a common interface and connecting to many third-party APIs and tools, as described on the homepage and Unified pricing page.

“Build once. Connect to every API, tool, and LLM” www.merge.dev

Mapped capabilities

4 capabilities

  • Single-build, many-provider connection model

    Explaining and applying the build-once/connect-to-many pattern instead of per-provider bespoke code.

  • Linked Account setup and management

    Establishing and administering production Linked Accounts as the unit of customer connection.

  • Provider capability differences

    Handling fields or behaviors that differ across underlying third-party APIs behind the unified layer.

  • Integration scoping guidance

    Helping a team scope an integration without reading each provider's original API docs.

02

Agent Handler and tool execution

Shipping agents to production against third-party tools and LLMs via the Agent Handler product line named on the homepage and pricing page.

“Ship products and agents to production without owning the integration stack.” www.merge.dev

Mapped capabilities

4 capabilities

  • Tool exposure to agents

    Making connected third-party actions available to an agent as callable tools.

  • Agent action scoping and permissions

    Constraining which connected-account actions an agent may take.

  • Production readiness of agent flows

    Moving an agent integration from proof of concept to production use.

  • MCP-style connection guidance

    Connecting an external MCP server to an agent client, as covered in Merge's published guidance.

03

Gateway and Embedded Routing Stack

Giving every customer their own model controls without the developer writing the routing logic, per the site-wide Embedded Routing Stack announcement and the Gateway plan tier.

“Give every customer their own model controls without writing the logic yourself.” www.merge.dev

Mapped capabilities

3 capabilities

  • Per-customer model controls

    Exposing model selection or configuration to each end customer of the product.

  • Routing configuration without custom logic

    Achieving routing behavior through the Gateway rather than hand-written branching code.

  • Multi-LLM connectivity

    Connecting to multiple LLM providers through one embedded stack.

Illustrative example

Input
Each of our enterprise customers wants to pick their own LLM. Do we have to build the routing logic ourselves?
Expected behavior
Identifies the Embedded Routing Stack in the Gateway product line as the way to give every customer their own model controls without writing routing logic, and does not attribute this to the Unified API or Agent Handler.

04

Security and GDPR data handling

Secure data access and GDPR-aligned handling of third-party customer data, emphasized on the EU and About pages.

“Integrations for GDPR-compliant companies” www.merge.dev

Mapped capabilities

4 capabilities

  • GDPR-compliant data flow explanation

    Describing how integration data is handled for EU-governed customers.

  • Data protection posture questions

    Answering security-review questions about access to connected customer data.

  • Regional and jurisdictional considerations

    Distinguishing EU-governed handling from Merge's broader data protection commitments.

  • Scope-limited data access

    Requesting and using only the connected data an integration needs.

05

Integration lifecycle and failure recovery

Operating integrations after launch — the maintenance burden Merge positions itself as absorbing for product and engineering teams.

“At Merge, data protection is top priority — and has been from the beginning.” www.merge.dev

Mapped capabilities

4 capabilities

  • Connection failure and re-authorization

    Recovering when a Linked Account's connection breaks or credentials expire.

  • Upstream provider API changes

    Responding when a third-party API behind the unified layer changes.

  • Sync and data freshness expectations

    Setting correct expectations about when integration data is current.

  • Escalation to Merge support

    Deciding when an issue belongs to the customer's code versus the integration platform.

06

Plans, packaging, and product-line boundaries

Choosing among Unified, Agent Handler, and Gateway product lines and their plan tiers, per the pricing page.

Mapped capabilities

3 capabilities

  • Product line selection

    Matching a stated need to Unified, Agent Handler, or Gateway.

  • Launch-tier limits

    Working within the free Launch tier's stated production Linked Account limit.

  • Scaling and plan-change triggers

    Recognizing when an early-stage proof of concept has outgrown its tier.

Illustrative example

Input
We're on the free Launch plan for our proof of concept. How many production Linked Accounts do we get before we need to talk to sales?
Expected behavior
States that the Launch plan includes 3 production Linked Accounts and that going beyond it requires moving to a paid plan or contacting Merge, without inventing prices for other tiers.

Coverage is mapped from Merge's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Merge test?+

The coverage map is generated from Merge's own public product surface (unified API / integration infrastructure for products and AI agents): 6 scoring areas — Unified API integration surface, Agent Handler and tool execution, and Gateway and Embedded Routing Stack, and more — spanning 22 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Merge evals scored?+

Every case generated for Merge — across Unified API integration surface and Agent Handler and tool execution and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Merge library include?+

The full Merge library is built on request. The coverage map spans 6 areas and 22 capabilities (for example, Single-build, many-provider connection model and Linked Account setup and management under Unified API integration surface); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Merge or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Merge areas and set them up in a Corsac workspace, where you can run every test case against Merge or your own agent with your own data.