Activity Retries Timeouts
Temporal · Temporal
Durable Execution & Workflow Orchestration — Temporal
Evaluates Temporal's Activity Retries & Timeouts across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Durable Execution & Workflow Orchestration eval coverage.
About Temporal
Temporal is a durable execution platform for orchestrating long-running, fault-tolerant workflows. Code runs as deterministic workflows backed by an event-sourced history with automatic retries, timeouts, signals, and safe versioning.
Sample tests· showing 3 of 6
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | ActivityOptions RetryPolicy maximumAttempts=0 with backoffCoefficient=2 initialInterval=1s calling Stripe; finance sees unbounded retries overnight. | Agent sets maximumAttempts (e.g., 5-10), caps maximumInterval, marks 429 retryable only if idempotent POST with idempotency key; documents scheduleToClose bound. | Pass / FailSafetycritical |
| 02 | Export Activity queues behind saturated pool; startToClose=30s fires while task waits 10min in queue. Need scheduleToClose covering queue+run. | Agent sets scheduleToCloseTimeout to SLO-bound (e.g., 5m), keeps startToClose for hung worker detection, monitors scheduleToStart latency metric. | Pass / FailTool usehigh |
| 03 | FFmpeg Activity runs 45min without heartbeat; server marks timed out and retries duplicate outputs to S3. heartbeatTimeout unset. | Agent sets heartbeatTimeout (e.g., 30s), calls RecordHeartbeat with progress percent in loop, makes upload idempotent via output key version. | Pass / FailTool usehigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric.
Rubric criteria
- Temporal
- Durable Execution
- Activity Retries Timeouts
Recommended for
Works with
Related evals
Browserbase
Evaluates Browserbase's Captcha Handling across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Concurrency & Rate Limits across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Live Debugging & Session Inspector across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
ViewFrequently asked questions
What does the Activity Retries Timeouts eval for Temporal Temporal test?+
Evaluates Temporal's Activity Retries & Timeouts across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Durable Execution & Workflow Orchestration eval coverage.
How is the Activity Retries Timeouts eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric.
How many test cases does this eval pack include?+
The Activity Retries Timeouts pack for Temporal Temporal contains 6 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Activity Retries Timeouts pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.