SKILLoptionalHermes Optional Skills

Modal

Name: Modal
Author: Orchestra Research

Serverless GPU cloud for ML jobs and model APIs

InfrastructureServerlessGPUCloudDeploymentModalOptionalHermes skill

Last registry verification2026-08-18v1.0.1Orchestra Research

Open original source ↗اقرأ بالعربية

Plain meaning

What does it add to Hermes?

Serverless GPU cloud for ML jobs and model APIs

Modal is a skill related to servers and infrastructure. It helps the agent understand or operate technical resources that can affect cost, availability, and security.

This plain-language explanation is based on the publisher description. The original text remains visible for verification.

Use it when

Use it when your goal in servers and infrastructure is clear and you can limit it to the data and actions it actually needs.

Skip it when

Do not add it merely to experiment when Hermes already has a simpler path, or when you cannot review its source and permissions.

Who is it for?

Best for users who want a repeatable way of working inside Hermes.

Safe first test

Start in a test environment with a limited account and inspect status before creating or deleting anything.

Original publisher description

Serverless GPU cloud for ML jobs and model APIs

✓

Data source

This entry was indexed from Hermes Optional Skills. Our explanation interprets the type and domain without inventing a capability not present upstream.

Security review

The source is official or editorially reviewed, but you still need to review permissions and version compatibility.

Safe setup path

Inspect, install, then test.

01
Open the source
Match the publisher, license, and description to your need. Check the real update history.
02
Review permissions and secrets
Never paste a secret value into this site. Use environment-variable names and grant the smallest scope.
03
Copy setup only after review
The controls below copy text. They do not execute commands on your device.
04
Test with a non-sensitive task
Inspect the visible tools, then exclude write or delete tools you do not need.

Install command

Review the command, then copy it.

hermes skills install modal

Hermes Belarabi does not execute this command. Installation happens on your device and remains subject to Hermes scanning and your review.

The full skill definition

Exactly what Hermes loads when this skill runs.

Reproduced from the official documentation. Read it before enabling the skill: this text becomes the agent's instructions.

Serverless GPU cloud for ML jobs and model APIs.

Skill metadata

A lookup table. Do not read it all; find the row that applies to you.

Source	Optional — install with `hermes skills install official/mlops/modal`
Path	`optional-skills/mlops/modal`
Version	`1.0.1`
Author	Orchestra Research
License	MIT
Dependencies	`modal>=1.0`
Platforms	linux, macos, windows
Tags	`Infrastructure`, `Serverless`, `GPU`, `Cloud`, `Deployment`, `Modal`

Reference: full SKILL.md

Explains the idea itself. Read it slowly; the later sections build on it.

Guide to running ML workloads on Modal's serverless GPU cloud platform.

Quick start

Ordered, practical steps. Run one and confirm it worked before moving on.

Installation

Shell2 lines

pip install modal
modal setup  # Opens browser for authentication

Hello World with GPU

Python12 lines



app = modal.App("hello-gpu")

@app.function(gpu="T4")
def gpu_info():
    import subprocess
    return subprocess.run(["nvidia-smi"], capture_output=True, text=True).stdout

@app.local_entrypoint()
def main():
    print(gpu_info.remote())

Run: modal run hello_gpu.py

Basic inference endpoint

Python19 lines



app = modal.App("text-generation")
image = modal.Image.debian_slim().pip_install("transformers", "torch", "accelerate")

@app.cls(gpu="A10G", image=image)
class TextGenerator:
    @modal.enter()
    def load_model(self):
        from transformers import pipeline
        self.pipe = pipeline("text-generation", model="gpt2", device=0)

    @modal.method()
    def generate(self, prompt: str) -> str:
        return self.pipe(prompt, max_length=100)[0]["generated_text"]

@app.local_entrypoint()
def main():
    print(TextGenerator().generate.remote("Hello, world"))

Core concepts

A lookup table. Do not read it all; find the row that applies to you.

Key components

Component	Purpose
`App`	Container for functions and resources
`Function`	Serverless function with compute specs
`Cls`	Class-based functions with lifecycle hooks
`Image`	Container image definition
`Volume`	Persistent storage for models/data
`Secret`	Secure credential storage

Execution modes

Command	Description
`modal run script.py`	Execute and exit
`modal serve script.py`	Development with live reload
`modal deploy script.py`	Persistent cloud deployment

GPU configuration

A lookup table. Do not read it all; find the row that applies to you.

Available GPUs

GPU	VRAM	Best For
`T4`	16GB	Budget inference, small models
`L4`	24GB	Inference, Ada Lovelace arch
`A10G`	24GB	Training/inference, 3.3x faster than T4
`L40S`	48GB	Recommended for inference (best cost/perf)
`A100-40GB`	40GB	Large model training
`A100-80GB`	80GB	Very large models
`H100`	80GB	Fastest, FP8 + Transformer Engine
`H200`	141GB	Auto-upgrade from H100, 4.8TB/s bandwidth
`B200`	Latest	Blackwell architecture

GPU specification patterns

Python14 lines

# Single GPU
@app.function(gpu="A100")

# Specific memory variant
@app.function(gpu="A100-80GB")

# Multiple GPUs (up to 8)
@app.function(gpu="H100:4")

# GPU with fallbacks
@app.function(gpu=["H100", "A100", "L40S"])

# Any available GPU
@app.function(gpu="any")

Container images

Explains the idea itself. Read it slowly; the later sections build on it.

Python13 lines

# Basic image with pip
image = modal.Image.debian_slim(python_version="3.11").pip_install(
    "torch==2.1.0", "transformers==4.36.0", "accelerate"
)

# From CUDA base
image = modal.Image.from_registry(
    "nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04",
    add_python="3.11"
).pip_install("torch", "transformers")

# With system packages
image = modal.Image.debian_slim().apt_install("git", "ffmpeg").pip_install("whisper")

Persistent storage

Explains the idea itself. Read it slowly; the later sections build on it.

Python11 lines

volume = modal.Volume.from_name("model-cache", create_if_missing=True)

@app.function(gpu="A10G", volumes={"/models": volume})
def load_model():
    import os
    model_path = "/models/llama-7b"
    if not os.path.exists(model_path):
        model = download_model()
        model.save_pretrained(model_path)
        volume.commit()  # Persist changes
    return load_from_path(model_path)

Web endpoints

A lookup table. Do not read it all; find the row that applies to you.

FastAPI endpoint decorator

Python4 lines

@app.function()
@modal.fastapi_endpoint(method="POST")
def predict(text: str) -> dict:
    return {"result": model.predict(text)}

Full ASGI app

Python11 lines

from fastapi import FastAPI
web_app = FastAPI()

@web_app.post("/predict")
async def predict(text: str):
    return {"result": await model.predict.remote.aio(text)}

@app.function()
@modal.asgi_app()
def fastapi_app():
    return web_app

Web endpoint types

Decorator	Use Case
`@modal.fastapi_endpoint()`	Simple function → API
`@modal.asgi_app()`	Full FastAPI/Starlette apps
`@modal.wsgi_app()`	Django/Flask apps
`@modal.web_server(port)`	Arbitrary HTTP servers

Dynamic batching

Explains the idea itself. Read it slowly; the later sections build on it.

Python5 lines

@app.function()
@modal.batched(max_batch_size=32, wait_ms=100)
async def batch_predict(inputs: list[str]) -> list[dict]:
    # Inputs automatically batched
    return model.batch_predict(inputs)

Secrets management

Explains the idea itself. Read it slowly; the later sections build on it.

Shell2 lines

# Create secret
modal secret create huggingface HF_TOKEN=hf_xxx

Python4 lines

@app.function(secrets=[modal.Secret.from_name("huggingface")])
def download_model():
    import os
    token = os.environ["HF_TOKEN"]

Scheduling

Explains the idea itself. Read it slowly; the later sections build on it.

Python7 lines

@app.function(schedule=modal.Cron("0 0 * * *"))  # Daily midnight
def daily_job():
    pass

@app.function(schedule=modal.Period(hours=1))
def hourly_job():
    pass

Performance optimization

Explains the idea itself. Read it slowly; the later sections build on it.

Cold start mitigation

Python6 lines

# Modal 1.0 autoscaler params: scaledown_window (was container_idle_timeout).
# Input concurrency moved to the @modal.concurrent decorator.
@app.function(scaledown_window=300)  # Keep warm 5 min
@modal.concurrent(max_inputs=10)     # Handle concurrent requests per container
def inference():
    pass

Model loading best practices

Python9 lines

@app.cls(gpu="A100")
class Model:
    @modal.enter()  # Run once at container start
    def load(self):
        self.model = load_model()  # Load during warm-up

    @modal.method()
    def predict(self, x):
        return self.model(x)

Parallel processing

Explains the idea itself. Read it slowly; the later sections build on it.

Python10 lines

@app.function()
def process_item(item):
    return expensive_computation(item)

@app.function()
def run_parallel():
    items = list(range(1000))
    # Fan out to parallel containers
    results = list(process_item.map(items))
    return results

Common configuration

Explains the idea itself. Read it slowly; the later sections build on it.

Python12 lines

@app.function(
    gpu="A100",
    memory=32768,              # 32GB RAM
    cpu=4,                     # 4 CPU cores
    timeout=3600,              # 1 hour max
    scaledown_window=120,      # Keep warm 2 min (was container_idle_timeout)
    retries=3,                 # Retry on failure
    max_containers=10,         # Max concurrent containers (was concurrency_limit)
    min_containers=1,          # Keep N containers warm (was keep_warm)
)
def my_function():
    pass

Modal 1.0 autoscaler renames (see the migration guide ↗): - container_idle_timeout → scaledown_window - concurrency_limit → max_containers - keep_warm → min_containers - allow_concurrent_inputs=N → the @modal.concurrent(max_inputs=N) decorator

Debugging

A troubleshooting section. Find the symptom that matches yours rather than reading it end to end.

Python6 lines

# Test locally
if __name__ == "__main__":
    result = my_function.local()

# View logs
# modal app logs my-app

Common issues

A lookup table. Do not read it all; find the row that applies to you.

Issue	Solution
Cold start latency	Increase `scaledown_window`, use `@modal.enter()`
GPU OOM	Use larger GPU (`A100-80GB`), enable gradient checkpointing
Image build fails	Pin dependency versions, check CUDA compatibility
Timeout errors	Increase `timeout`, add checkpointing

References

Explains the idea itself. Read it slowly; the later sections build on it.

Advanced Usage ↗ - Multi-GPU, distributed training, cost optimization
Troubleshooting ↗ - Common issues and solutions

Resources

Explains the idea itself. Read it slowly; the later sections build on it.

Documentation: https://modal.com/docs
Examples: https://github.com/modal-labs/modal-examples
Pricing: https://modal.com/pricing
Discord: https://discord.gg/modal