Queue Webhooks Reliability
fal · fal
fal evals — Queue Webhooks Reliability (relift v3)
About fal
fal is a generative media inference platform offering fast, scalable image, video, and audio generation through a simple API. It hosts leading open models (FLUX, Stable Diffusion, Whisper) and supports fine-tuned LoRA routing, webhooks, and queue-based async generation at production scale.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Batch worker submitted 50 image jobs; must not fetch /response while status is IN_QUEUE. | Loop handler.status(with_logs=True) until Completed; only then handler.get(); persist request_id per job. | Pass / FailWorkflowhigh |
| 02 | Image-to-video model runs 8+ minutes; operator UI needs incremental log lines. | Use handler.iter_events(with_logs=True) and print InProgress logs until Completed; then get(). | Pass / FailWorkflowmedium |
| 03 | Audit pipeline cannot tolerate duplicate generated artifacts on 503 retry. | Pass headers={'X-Fal-No-Retry': '1'} on fal.queue.submit; on failure surface error without assuming retry. | Pass / FailPolicyhigh |
Rubric criteria
- Fal
- Ai Platform
- Queue Webhooks Reliability
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.