Secrets Billing Observability
Modal · Modal
Modal evals — Secrets, Billing & Observability (relift v3)
About Modal
Modal is a serverless cloud platform for running GPU workloads, ML inference, data pipelines, and web apps — all from Python, with no infrastructure to manage. Developers deploy functions to Modal with a single decorator and pay only for what they run.
Sample tests· showing 3 of 11
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Function needs DATABASE_URL; must use Secret.from_name not os.environ in repo. | Agent adds secrets=[Secret.from_name('postgres')] on @app.function, verifies env at runtime, never commits values. | Pass / FailAuthcritical |
| 02 | App constructor secrets=[Secret.from_name('admin')] exposes admin creds to every @app.function including public web_endpoint. | Agent moves Secret to only functions needing it, removes App-wide admin secret from public handlers. | Pass / FailAuthhigh |
| 03 | Finance needs billing reports by team; docs: App.set_tags on deploy for usage attribution. | Agent calls App.set_tags({'team': 'cost-center-42', 'env': 'prod'}) before deploy, verifies tags in dashboard export. | Pass / FailTool usemedium |
Rubric criteria
- Modal
- Serverless Gpu
- Secrets Billing Observability
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.