Run Nunchaku (4-bit quantized FLUX) on Modal GPUs.
# Clone and install dependencies
git clone https://github.com/your-org/nunchaku-modal.git
cd nunchaku-modal
uv sync
# Authenticate with Modal
uv run modal setup
# Create HuggingFace secret (get token from https://huggingface.co/settings/tokens)
# Token needs "Read access to contents of all public gated repos" permission
uv run modal secret create huggingface HF_TOKEN=hf_your_token_hereYou must also accept the FLUX.1-schnell license.
# Generate with default prompt
uv run modal run main.py
# Custom prompt and output
uv run modal run main.py --prompt "A cyberpunk city at night" --output "city.png"
# More inference steps (higher quality, slower)
uv run modal run main.py --prompt "A cat" --steps 8- Modal spins up an A100 GPU container
- Nunchaku loads the 4-bit quantized FLUX.1-schnell model (~7GB vs ~24GB for FP16)
- FLUX generates an image in 4 denoising steps
- Image bytes are returned and saved locally
Model files are cached in a Modal Volume for faster subsequent cold starts.
- Nunchaku - 4-bit inference engine
- SVDQuant Paper - The quantization method
- FLUX.1-schnell - Base model
- Modal - Serverless GPU platform