Failure Handling Compensation
Temporal · Temporal
Durable Execution & Workflow Orchestration — Temporal
Evaluates Temporal's Failure Handling & Compensation across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Durable Execution & Workflow Orchestration eval coverage.
About Temporal
Temporal is a durable execution platform for orchestrating long-running, fault-tolerant workflows. Code runs as deterministic workflows backed by an event-sourced history with automatic retries, timeouts, signals, and safe versioning.
Sample tests· showing 3 of 6
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Saga books flight, hotel, payment; hotel fails; must compensate payment then flight in reverse order. | Agent implements compensation stack (LIFO), each compensating Activity idempotent, workflow records completed forward steps. | Pass / FailSafetycritical |
| 02 | FraudCheck Activity returns retryable error on hard decline; wastes time and triggers alert fatigue. | Agent maps fraud decline to nonRetryable ApplicationError, workflow terminal path notifies customer, metrics tag terminal reason. | Pass / FailTool usehigh |
| 03 | REQUEST_CANCEL during Activity; lock held in Redis; need defer cleanup on ctx.Done(). | Agent implements Activity cancellation handler releasing lock, heartbeats until cleanup done, verifies no double release. | Pass / FailSafetyhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric.
Rubric criteria
- Temporal
- Durable Execution
- Failure Handling Compensation
Recommended for
Works with
Related evals
Browserbase
Evaluates Browserbase's Captcha Handling across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Concurrency & Rate Limits across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Live Debugging & Session Inspector across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
ViewFrequently asked questions
What does the Failure Handling Compensation eval for Temporal Temporal test?+
Evaluates Temporal's Failure Handling & Compensation across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Durable Execution & Workflow Orchestration eval coverage.
How is the Failure Handling Compensation eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric.
How many test cases does this eval pack include?+
The Failure Handling Compensation pack for Temporal Temporal contains 6 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Failure Handling Compensation pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.