Webhooks
Replicate · Replicate
AI Model Hosting — Replicate
Replicate evals — Webhooks (relift v3 InfraRed)
About Replicate
Replicate is an AI model-hosting platform — run thousands of community and custom Cog-packaged models (FLUX, SDXL, Llama, Whisper, custom fine-tunes) via a simple HTTP API with predictions, webhooks, streaming, deployments, and per-second billing.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Integrator sets webhook_events_filter=['completed'] expecting only one delivery per prediction. | Choose the smallest filter subset that satisfies downstream needs: ['completed'] for fire-and-forget terminal delivery; ['start','output','completed'] for progress UIs. Larger filters mean more deliveries → more HMAC verifications → more retry exposure. Document the chosen filter in the operator ru… | Pass / FailAi Platformmedium |
| 02 | Webhook delivery carries headers webhook-id, webhook-timestamp, webhook-signature. Receiver wants to validate authenticity. | Compute HMAC-SHA256 over the canonical string '<webhook-id>.<webhook-timestamp>.<raw-body>' with the per-endpoint signing secret. Compare in constant time against the v1 entry in webhook-signature. Reject if signature mismatches or webhook-timestamp is outside a tolerance window (e.g., 5 minutes) t… | Pass / FailAi Platformcritical |
| 03 | Webhook handler does heavy work inline (re-encodes a 4K video) and responds 200 after 45 seconds. | Return 2xx as fast as possible (typically <5s) — push real work to a background queue. A slow handler risks Replicate timing out the delivery and retrying, causing duplicate processing. The exact delivery deadline [REQUIRES-VERIFICATION] — design for sub-second response regardless. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Replicate
- Ai Platform
- Webhooks
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.