Eval Library
A
For AssemblyAIAI Platform

Transcript Features

AssemblyAI (Universal-2 + LeMUR) · AssemblyAI

Speech AI Platform — AssemblyAI

AssemblyAI evals — Transcript Features (relift v3 InfraRed)

About AssemblyAI

AssemblyAI is a speech-AI platform with Universal-2 speech-to-text, real-time streaming, Speaker Diarization, Audio Intelligence (summarization, sentiment, content moderation), and LeMUR — an LLM framework that runs over transcripts (task, summary, question-answer, action items).

Employees

~150

Industry

Speech AI

Headquarters

San Francisco, CA

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Downstream subtitling pipeline expects punctuated/cased text. Agent sets punctuate=false to save tokens; receives a wall of lowercase words.

Default punctuate=true and format_text=true for human-readable transcripts. Disable only for downstream NLP that re-tokenizes (e.g., custom punctuation models). Verify text shape on a few samples before pinning the choice.

Pass / FailAi Platformmedium
02

Researcher needs verbatim transcripts including 'um', 'uh', false starts for qualitative coding. Sets disfluencies=true.

disfluencies=true preserves um/uh/false-starts in the transcript. For clean subtitles set disfluencies=false (default). Document the choice in your data schema so consumers know whether disfluencies are present.

Pass / FailAi Platformmedium
03

Family-friendly podcast pipeline sets filter_profanity=true expecting masked expletives in the transcript text.

filter_profanity=true replaces profanity with masked tokens in transcript.text (typical mask: f***). Audio is unchanged; redaction is text-only. For audio removal use redact_pii_audio (different surface). Verify the mask shape on sample utterances.

Pass / FailAi Platformmedium

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Assemblyai
  • Ai Platform
  • Transcript Features

Recommended for

AssemblyAI (Universal-2 + LeMUR)AssemblyAI customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.