Workflow Determinism Replay
Temporal · Temporal
Durable Execution & Workflow Orchestration — Temporal
Evaluates Temporal's Workflow Determinism & Replay across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Durable Execution & Workflow Orchestration eval coverage.
About Temporal
Temporal is a durable execution platform for orchestrating long-running, fault-tolerant workflows. Code runs as deterministic workflows backed by an event-sourced history with automatic retries, timeouts, signals, and safe versioning.
Sample tests· showing 3 of 6
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | TypeScript OrderWorkflow calls `Date.now()` to pick shipping tier; Worker logs NondeterminismError on replay after pod restart. Event History already recorded a timer branch. Docs require workflow.Now() or SideEffect. | Agent removes raw Date.now(), uses workflow.Now() (or SideEffect for one-time external clock read), adds replay test in CI, and redeploys Worker with same Task Queue. | Pass / FailTool usecritical |
| 02 | Go BillingWorkflow reads LaunchDarkly at start using SideEffect wrapper; flag must not re-fetch on replay changing branch. History shows Activity scheduled later — flag choice must be pinned. | Agent wraps external read in workflow.SideEffect (Go) or equivalent, stores result in workflow variable, documents that Activities are for mutating side effects not one-off reads. | Pass / FailTool usehigh |
| 03 | Python onboarding workflow uses random.shuffle on candidate list; replay after worker crash fails. Docs: workflow.random().random() or deterministic seed from workflow info. | Agent replaces Math.random/random.shuffle with SDK deterministic random API, adds unit replay test with history fixture, verifies same order on replay. | Pass / FailTool usehigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric.
Rubric criteria
- Temporal
- Durable Execution
- Workflow Determinism Replay
Recommended for
Works with
Related evals
Browserbase
Evaluates Browserbase's Captcha Handling across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Concurrency & Rate Limits across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
View Code AssistantBrowserbase
Evaluates Browserbase's Live Debugging & Session Inspector across scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Browser infrastructure eval coverage.
ViewFrequently asked questions
What does the Workflow Determinism Replay eval for Temporal Temporal test?+
Evaluates Temporal's Workflow Determinism & Replay across 6 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Durable Execution & Workflow Orchestration eval coverage.
How is the Workflow Determinism Replay eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric.
How many test cases does this eval pack include?+
The Workflow Determinism Replay pack for Temporal Temporal contains 6 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Workflow Determinism Replay pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.