Transcript Features
AssemblyAI (Universal-2 + LeMUR) · AssemblyAI
Speech AI Platform — AssemblyAI
AssemblyAI evals — Transcript Features (relift v3 InfraRed)
About AssemblyAI
AssemblyAI is a speech-AI platform with Universal-2 speech-to-text, real-time streaming, Speaker Diarization, Audio Intelligence (summarization, sentiment, content moderation), and LeMUR — an LLM framework that runs over transcripts (task, summary, question-answer, action items).
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Downstream subtitling pipeline expects punctuated/cased text. Agent sets punctuate=false to save tokens; receives a wall of lowercase words. | Default punctuate=true and format_text=true for human-readable transcripts. Disable only for downstream NLP that re-tokenizes (e.g., custom punctuation models). Verify text shape on a few samples before pinning the choice. | Pass / FailAi Platformmedium |
| 02 | Researcher needs verbatim transcripts including 'um', 'uh', false starts for qualitative coding. Sets disfluencies=true. | disfluencies=true preserves um/uh/false-starts in the transcript. For clean subtitles set disfluencies=false (default). Document the choice in your data schema so consumers know whether disfluencies are present. | Pass / FailAi Platformmedium |
| 03 | Family-friendly podcast pipeline sets filter_profanity=true expecting masked expletives in the transcript text. | filter_profanity=true replaces profanity with masked tokens in transcript.text (typical mask: f***). Audio is unchanged; redaction is text-only. For audio removal use redact_pii_audio (different surface). Verify the mask shape on sample utterances. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Assemblyai
- Ai Platform
- Transcript Features
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.