Eval Library
HA
For Hume AIAI Platform

Hume Webhooks Batch Jobs

Hume AI (EVI / Octave) · Hume AI

Empathic Voice AI — Hume AI

Hume AI evals — Webhooks & Post-call / Batch Jobs (relift v3 InfraRed)

About Hume AI

Hume AI builds empathic voice technology — the Empathic Voice Interface (EVI), an expression-measurement API across face, voice, and language, and the Octave text-to-speech model — for emotionally aware, responsibly governed voice experiences.

Employees

~60

Industry

Voice AI

Headquarters

New York, NY

Website

hume.ai

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Operator submits a batch Expression Measurement job over 500 recordings and needs to map each result back to its source recording.

Submit via POST /v0/batch/jobs and persist the returned job id; map each input (file/url) to its own stable application-side identifier so predictions can be reattached to source rows. Do not rely on input ordering alone for identity.

Pass / FailAi Platformhigh
02

A single batch job is submitted with a very large media file (or hundreds of long files) and the operator assumes it will always be accepted.

Pre-check inputs against documented per-file size/duration and per-job limits and split oversized work into multiple jobs; on a limit rejection, do not retry the identical payload. [REQUIRES-VERIFICATION] for the exact numeric caps.

Pass / FailAi Platformmedium
03

After submit, the operator polls GET /v0/batch/jobs/{id} every 500ms until COMPLETED.

Poll job status with exponential backoff and rely on the terminal states (COMPLETED / FAILED). Do not hammer the status endpoint sub-second, and do not assume IN_PROGRESS means failure.

Pass / FailAi Platformmedium

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3. Emotion-expression scores are probabilistic perceived-expression signals, not ground-truth affect.

Rubric criteria

  • Hume
  • Ai Platform
  • Webhooks Batch Jobs

Recommended for

Hume AI (EVI / Octave)Hume AI customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.