Eval Library
L
For LiveKitAI Platform

Cloud Vs Self Host And Scaling

LiveKit (Cloud + Agents) · LiveKit

Real-time Voice & Video Infra — LiveKit

LiveKit evals — Cloud vs Self-host & Scaling (relift v3 InfraRed)

About LiveKit

LiveKit is open-source real-time voice/video infrastructure used to build voice agents and live experiences — a WebRTC SFU, telephony (SIP), recording/egress, and the LiveKit Agents framework for STT→LLM→TTS pipelines, available as LiveKit Cloud and self-hosted.

Employees

~50

Industry

Voice AI Infrastructure

Headquarters

New York, NY

Website

livekit.io

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Operator wants the cheapest 30-participant beta with minimal ops; later expects to scale to 5000-room production.

LiveKit Cloud provides the SFU + edge mesh + SIP + egress as a managed service with regions; self-host gives full control on infrastructure cost but requires k8s, Redis, egress workers, TURN/TLS, observability, and capacity planning. Recommend Cloud for the beta and revisit only if cost or data-res…

Pass / FailAi Platformmedium
02

Operator runs livekit-server on two nodes behind a load balancer with no shared state. Participants on different nodes can't see each other.

Multi-node self-host requires Redis (or compatible) for room/participant coordination so signaling on any node sees the global room state. Configure redis: {address, password} in the server config on every node. Single-node deployments can skip Redis but cap at the node's capacity.

Pass / FailAi Platformcritical
03

Self-host: operator wants SIP telephony but only deployed livekit-server.

SIP is a separate worker (livekit-sip) that registers with the server and handles SIP signaling/media to carriers. Deploy with the SIP listener exposed to the trunk provider over a stable IP/UDP. Do not expose SIP directly from the SFU node — separate concerns and security boundaries.

Pass / FailAi Platformhigh

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Livekit
  • Ai Platform
  • Cloud Vs Self Host And Scaling

Recommended for

LiveKit (Cloud + Agents)LiveKit customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.