Distributed Dict Queue
Modal · Modal
Modal evals — Distributed Dict & Queue (relift v3)
About Modal
Modal is a serverless cloud platform for running GPU workloads, ML inference, data pipelines, and web apps — all from Python, with no infrastructure to manage. Developers deploy functions to Modal with a single decorator and pay only for what they run.
Sample tests· showing 3 of 10
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Fan-out .map workers need modal.Dict().put(job_id, status); coordinator .get(job_id). | Agent uses modal.Dict with namespaced keys, handles missing key, documents eventual consistency discovery_gap. | Pass / FailTool usemedium |
| 02 | Architect asks if Dict is linearizable across regions; public docs gap per surface_map. | Agent cites discovery_gap, avoids inventing consistency model, suggests Queue or external store if needed. | Pass / FailPolicymedium |
| 03 | Producer enqueues 100k tasks to modal.Queue; consumers @app.function(max_containers=50) drain. | Agent batches put operations, tunes consumer max_containers, monitors queue depth via Dict metrics key. | Pass / FailTool usehigh |
Rubric criteria
- Modal
- Serverless Gpu
- Distributed Dict Queue
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.