Skip to content

About

Example of running Nunchaku on Modal

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

nunchaku-modal

Run Nunchaku (4-bit quantized FLUX) on Modal GPUs.

Setup

# Clone and install dependencies
git clone https://github.com/your-org/nunchaku-modal.git
cd nunchaku-modal
uv sync

# Authenticate with Modal
uv run modal setup

# Create HuggingFace secret (get token from https://huggingface.co/settings/tokens)
# Token needs "Read access to contents of all public gated repos" permission
uv run modal secret create huggingface HF_TOKEN=hf_your_token_here

You must also accept the FLUX.1-schnell license.

Usage

# Generate with default prompt
uv run modal run main.py

# Custom prompt and output
uv run modal run main.py --prompt "A cyberpunk city at night" --output "city.png"

# More inference steps (higher quality, slower)
uv run modal run main.py --prompt "A cat" --steps 8

How it works

  1. Modal spins up an A100 GPU container
  2. Nunchaku loads the 4-bit quantized FLUX.1-schnell model (~7GB vs ~24GB for FP16)
  3. FLUX generates an image in 4 denoising steps
  4. Image bytes are returned and saved locally

Model files are cached in a Modal Volume for faster subsequent cold starts.

References

About

Example of running Nunchaku on Modal

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages