Volumes Image Build Cache

Trainer writes checkpoints to mounted modal.Volume at /checkpoints on @app.function(volumes={...}). Evaluator on second function reads missing latest.pt; writer never called volume.commit().

Agent adds volume.commit() after each checkpoint write, documents reader must volume.reload() before read, and verifies cross-function visibility.

Parallel map workers write metrics.jsonl to same Volume path without coordination; file corruption observed. Docs warn about concurrent writers; pattern uses modal.Queue or per-worker paths.

Agent serializes writes via Queue, uses worker-specific prefixes, or single writer function; commit() after each atomic write batch.

Serving @app.function reads /models/weights from Volume mounted read-only. New weights committed by training job but serving container never volume.reload(); stale inference.

Agent adds volume.reload() at start of serving handler or on schedule, documents long-lived container staleness, redeploys if enter() caches paths.