For DeepgramAI PlatformTranscription Accuracy

Batch Stt Async Callbacks

Deepgram · Deepgram

Speech AI Platform — Deepgram

Deepgram evals — Batch STT & Async Callbacks (relift v3)

About Deepgram

Deepgram is a speech-AI platform offering streaming and batch speech-to-text (Nova), Aura text-to-speech, speaker diarization, redaction, and smart formatting across 30+ languages — used by voice-agent platforms, contact centers, and media teams.

Employees

~150

Industry

Speech AI

Headquarters

San Francisco, CA

Website

deepgram.com

Sample tests· showing 3 of 8

#	Input	Expected behavior	Check
01	Async transcription for 45-minute podcast; customer server flaky during deploy; Deepgram retries callback delivery.	Accept initial 200 with request_id; implement idempotent webhook handler keyed by request_id; expect up to 10 retries per docs; persist transcript once successfully.	Pass / FailAi Platformhigh
02	POST /v1/listen with callback returns JSON containing request_id; later webhook POST includes same identifier.	Store request_id from immediate API response; join callback transcript payload to job row via request_id; surface in ops dashboard.	Pass / FailAi Platformmedium
03	callback URL is public HTTPS; security wants authentication on inbound transcript delivery [REQUIRES-VERIFICATION on signed callback headers].	Terminate TLS; validate source IP or shared secret if documented; reject unsigned unexpected POSTs; never expose callback URL without auth layer.	Pass / FailAi Platformcritical
Use this eval

How this eval is graded

Grade the agent's response against expected.ideal_behavior and expected.rubric. Score 1-5 per criterion (4+ passes; no criterion below 3). Penalize failure_modes. For safety outcomes, correct behavior is refuse/abstain or redact per policy.