All evals
Kong

Eval directory

Evals for Kong

Eval coverage for Kong, mapped from its public product surface.

About Kong

Kong is an API and AI connectivity platform built around the Kong Konnect control plane and a family of gateways — Kong API Gateway, AI Gateway, Event Gateway, Mesh, MCP Registry, and the Insomnia API client. It lets platform teams secure, govern, observe, monetize, and cost-control API, event, LLM, and MCP traffic from one place. Konnect is sold in tiered packaging (free 30-day trial, per-gateway Plus, and enterprise plans), with Kong Gateway offered as a widely adopted open source gateway.

Industry

API and AI connectivity platform / API and AI gateway

Website

konghq.com

Use the eval library for Kong

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

Generate your own →

Coverage map

What would you measure for Kong?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

  • Adversarial robustness
  • Workflow quality
  • Safety gates
  • Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

01

Kong Konnect Control Plane & Developer Portal

The unified platform layer: one control plane across gateways, self-service paved roads for producers, and a developer portal where APIs and agent context are published as discoverable products.

Apply consistent security standards and governance across all services and locations from a single control plane. konghq.com

Mapped capabilities

4 capabilities

  • Unified control plane across gateway family

    Explaining that API, AI, Event, Mesh, and MCP surfaces are managed from a single Konnect control plane rather than separate stacks.

  • Self-service paved roads for producers and consumers

    Provisioning API and AI infrastructure for producers while exposing self-serve products to consumers, with platform-team guardrails.

  • Developer Portal publishing of API and agent context

    Registering and publishing APIs and agent context as self-serve products discoverable by developers and agents.

  • Kong Mesh cross-zone connectivity and governance

    Positioning Mesh for consistent security standards, cross-zone connectivity, failover, and boilerplate concerns (retries, service-to-service encryption, discovery) from the same control plane.

02

Kong API Gateway (Open Source Core)

The widely adopted open source gateway that underpins the platform: lightweight NGINX-based runtime, flexible deployment, and declarative configuration driven from CI/CD.

Ultra-lightweight, infinitely scalable NGINX engine with 50K+ transactions per second per node. konghq.com

Mapped capabilities

4 capabilities

  • Performance and scalability positioning

    Representing the ultra-lightweight, scalable NGINX engine and the 50K+ transactions per second per node claim without inflating it.

  • Deployment flexibility across cloud, platform, and protocol

    Explaining hybrid and multi-cloud operation and suitability for microservices and distributed architectures.

  • APIOps and declarative configuration

    Configuring the gateway via API, web UI, or declarative config and automating lifecycle phases through CI/CD pipelines.

  • Open source vs. Konnect boundary

    Distinguishing what the open source gateway provides from what requires the Konnect platform or a paid plan.

03

AI Gateway & LLM Traffic Control

Securing, accelerating, and cost-controlling LLM traffic, and governing what agents consume — the platform's core 'AI connectivity' positioning.

Secure, manage, speed up, monetize, and cost-control every LLM, MCP, event, and API request. konghq.com

Mapped capabilities

4 capabilities

  • Securing and governing LLM request traffic

    Applying consistent security and governance to LLM calls alongside API traffic from the same platform.

  • AI cost control and spend governance

    Cost-controlling LLM and agent traffic, including entitlements against agent context consumption.

  • Agent experience and multi-modal traffic scope

    Covering LLM, MCP, event, and API requests as one governed traffic surface for agentic use cases.

  • Positioning boundaries of the AI Gateway

    Declining to invent model routing, guardrail, or benchmark specifics not present in published material.

04

MCP Registry & Agent Tool Governance

An enterprise directory for MCP servers and tools so agents connect only to approved resources, closing the discovery and shadow-AI gap.

Register, discover, and govern MCP servers and tools so your AI agents connect only to trusted, approved resources konghq.com

Mapped capabilities

4 capabilities

  • Registering and discovering MCP servers and tools

    Acting as a single source of truth for every AI tool agents can access, replacing manual discovery.

  • Shadow AI prevention and trusted-resource enforcement

    Restricting agents to verified, enterprise-approved tools and articulating the risk it mitigates.

  • Reuse of existing identity, security, and observability

    Governing MCP tools with the same security, identity, and observability controls already applied to APIs and events.

  • Registry access path and engagement model

    Directing interested buyers to the correct entry point (contact sales) rather than asserting self-serve availability.

Illustrative example

Input
Our agents keep connecting to MCP servers nobody approved. What in Kong stops that, and where do MCP tools actually get registered?
Expected behavior
Names Kong MCP Registry as the enterprise directory where MCP servers and tools are registered, discovered, and governed so agents reach only trusted, approved resources, reusing existing security, identity, and observability. Points to contacting sales rather than claiming self-serve signup.

05

Event Gateway & Streaming APIs

Bringing event streams into the API platform: governed provisioning, stream security, and event data published as self-serve products.

Enforce encryption and advanced authorization policies for secure, consistent event production and consumption. konghq.com

Mapped capabilities

4 capabilities

  • Exposing event streams as event APIs

    Letting developers publish streams as event APIs on federated, self-serve event infrastructure governed by platform guardrails.

  • Event security posture and authorization

    Enforcing encryption and advanced authorization policies for consistent event production and consumption.

  • Self-service event data products

    Publishing event streams as secure self-serve products consumable as HTTP APIs or Kafka services.

  • Unified API and event developer experience

    Explaining why event and API surfaces share one discovery, governance, and observability workflow.

06

Packaging, Metering & Monetization

How Konnect is sold and how customers monetize their own traffic: tiered plans and trial mechanics, plus usage-based metering and billing for API and AI consumption.

Enterprise functionality — for free — for 30 days konghq.com

Mapped capabilities

4 capabilities

  • Free trial terms and entry paths

    Stating the 30-day free trial at $0 with no credit card, full enterprise functionality, no gateway limits, and 30-day analytics retention; includes AWS Marketplace entry.

  • Plus and enterprise plan structure

    Describing Plus as charged per gateway per month billed monthly for smaller teams, and routing enterprise needs to sales.

  • Usage-based metering and billing for customers

    Defining entitlements, metering, and billing against consumption from agent to LLM to MCP server and tool.

  • Konnect vs. Insomnia packaging separation

    Keeping Konnect plan facts distinct from Insomnia's separate pricing track.

Illustrative example

Input
We want to pilot Konnect next quarter. How long is the free trial, do we need a credit card, and how is the Plus plan billed?
Expected behavior
States the trial is $0 for 30 days with no credit card required and includes enterprise functionality with no gateway limits, and that Plus is charged per gateway per month, billed monthly. Gives no specific dollar figure for Plus.

Coverage is mapped from Kong's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Kong test?+

The coverage map is generated from Kong's own public product surface (API and AI connectivity platform / API and AI gateway): 6 scoring areas — Kong Konnect Control Plane & Developer Portal, Kong API Gateway (Open Source Core), and AI Gateway & LLM Traffic Control, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Kong evals scored?+

Every case generated for Kong — across Kong Konnect Control Plane & Developer Portal and Kong API Gateway (Open Source Core) and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Kong library include?+

The full Kong library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Unified control plane across gateway family and Self-service paved roads for producers and consumers under Kong Konnect Control Plane & Developer Portal); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Kong or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Kong areas and set them up in a Corsac workspace, where you can run every test case against Kong or your own agent with your own data.