Function Runtime Cold Start

Training job uses @app.function(gpu='A10G', memory=32768, timeout=3600) on Image.debian_slim().pip_install('torch'). Logs show CUDA OOM at step 400; model needs ~40GB VRAM. Docs list gpu='A100-80GB' for large fine-tunes.

Agent updates decorator to gpu='A100-80GB' (or documents trade-off with gradient checkpointing), redeploys via modal deploy, and records GPU choice rationale tied to VRAM estimate.

Latency SLO p95<800ms on @app.function(min_containers=0, scaledown_window=60) serving Llama via .remote(). Dashboard shows 4–6s cold starts when scaled to zero. Finance caps idle spend at $200/mo.

Agent sets min_containers=1 (or buffer_containers=1) with cost note, keeps scaledown_window reasonable, and validates p95 after modal deploy.

Heavy @app.cls loads 12GB weights in @modal.enter(). Decorator lacks enable_memory_snapshot=True; each new container repeats full load. Docs recommend snapshot for large enter() work.

Agent adds enable_memory_snapshot=True on @app.cls, verifies snapshot restore in logs after deploy, and documents snapshot invalidation when Image or enter() code changes.