Eval Library
FA
For falAI Platform

Queue Webhooks Reliability

fal · fal

fal evals — Queue Webhooks Reliability (relift v3)

About fal

fal is a generative media inference platform offering fast, scalable image, video, and audio generation through a simple API. It hosts leading open models (FLUX, Stable Diffusion, Whisper) and supports fine-tuned LoRA routing, webhooks, and queue-based async generation at production scale.

Employees

~30

Industry

Generative AI Inference

Headquarters

San Francisco, CA

Website

fal.ai

Sample tests· showing 3 of 10

#InputExpected behaviorCheck
01

Batch worker submitted 50 image jobs; must not fetch /response while status is IN_QUEUE.

Loop handler.status(with_logs=True) until Completed; only then handler.get(); persist request_id per job.

Pass / FailWorkflowhigh
02

Image-to-video model runs 8+ minutes; operator UI needs incremental log lines.

Use handler.iter_events(with_logs=True) and print InProgress logs until Completed; then get().

Pass / FailWorkflowmedium
03

Audit pipeline cannot tolerate duplicate generated artifacts on 503 retry.

Pass headers={'X-Fal-No-Retry': '1'} on fal.queue.submit; on failure surface error without assuming retry.

Pass / FailPolicyhigh

Rubric criteria

  • Fal
  • Ai Platform
  • Queue Webhooks Reliability

Recommended for

falfal customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.