Add Dockerized CPU deployment and Gradio UI configuration

.dockerignore

@@ -0,0 +1,22 @@
+.git
+.github
+.agents
+.codex
+.venv
+__pycache__
+*.pyc
+*.wav
+*.pt
+*.ckpt
+*.safetensors
+.ipynb_checkpoints
+docs
+examples
+outputs
+results
+/data
+hf-cache
+compose*.yaml
+docker-compose*.yml
+Dockerfile
+README-docker.md

Dockerfile.cpu

@@ -0,0 +1,47 @@
+# syntax=docker/dockerfile:1
+
+FROM python:3.11-slim-bookworm
+
+ARG UV_VERSION=0.10.4
+
+ENV PYTHONUNBUFFERED=1 \
+    PIP_NO_CACHE_DIR=1 \
+    UV_LINK_MODE=copy \
+    UV_TORCH_BACKEND=cpu \
+    UV_HTTP_TIMEOUT=300 \
+    UV_HTTP_RETRIES=5 \
+    HF_HOME=/cache/huggingface \
+    GRADIO_ANALYTICS_ENABLED=false
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    ffmpeg \
+    libsndfile1 \
+    libgomp1 \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+RUN pip install --no-cache-dir "uv==${UV_VERSION}"
+
+COPY pyproject.toml README.md LICENSE ./
+COPY uv.cpu.lock ./uv.lock
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --frozen --no-dev --no-install-project
+
+COPY omnivoice ./omnivoice
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --frozen --no-dev
+
+ENV PATH="/app/.venv/bin:$PATH"
+
+EXPOSE 8001
+
+CMD ["omnivoice-demo", \
+     "--model", "k2-fsa/OmniVoice", \
+     "--ip", "0.0.0.0", \
+     "--port", "8001", \
+     "--device", "cpu", \
+     "--no-asr"]
+

README-docker.md

@@ -0,0 +1,142 @@
+# OmniVoice Docker deployment
+
+OmniVoice has separate reproducible CPU and CUDA image workflows:
+
+- `compose.yaml` contains the common port, cache volume, health check, and CUDA
+  image defaults.
+- `compose.cpu.yaml` selects `Dockerfile.cpu`, the `omnivoice-ui:cpu` image,
+  and `--device cpu`. It does not request a GPU.
+- `compose.gpu.yaml` retains the CUDA image from `Dockerfile` and adds an
+  NVIDIA GPU reservation.
+
+`uv.cpu.lock` selects CPU-only PyTorch packages. The CPU image must not contain
+CUDA or NVIDIA runtime packages. The regular `uv.lock` intentionally selects
+the CUDA 12.8 PyTorch build, so the GPU image is expected to contain large
+CUDA/NVIDIA dependencies.
+
+## Prerequisites
+
+- Docker Engine or Docker Desktop with Docker Compose v2 (`docker compose`).
+- Network access while building and during the first model download.
+- Enough disk space for the selected image and the model cache.
+- On Windows, Docker Desktop running with WSL 2 integration enabled.
+
+An NVIDIA deployment also requires a supported NVIDIA GPU, a sufficiently
+recent NVIDIA driver, and NVIDIA Container Toolkit or compatible Docker
+Desktop/WSL 2 GPU support.
+
+## Validate Compose configuration
+
+```bash
+# Common configuration
+docker compose -f compose.yaml config
+
+# CPU configuration
+docker compose -f compose.yaml -f compose.cpu.yaml config
+
+# NVIDIA configuration
+docker compose -f compose.yaml -f compose.gpu.yaml config
+```
+
+## CPU-only image
+
+Build only the CPU image:
+
+```bash
+docker compose --progress=plain -f compose.yaml -f compose.cpu.yaml build
+```
+
+Run packaging smoke tests without loading or downloading the OmniVoice model:
+
+```bash
+docker image inspect omnivoice-ui:cpu --format "{{.Created}} {{.Size}}"
+
+docker run --rm --entrypoint omnivoice-demo omnivoice-ui:cpu --help
+
+docker run --rm --entrypoint python omnivoice-ui:cpu -c \
+  "import torch, torchaudio, gradio, omnivoice; print('Imports OK'); print('PyTorch:', torch.__version__); print('TorchAudio:', torchaudio.__version__); print('CUDA build:', torch.version.cuda); print('CUDA available:', torch.cuda.is_available())"
+```
+
+For the CPU image, `torch.version.cuda` must be `None` and
+`torch.cuda.is_available()` must be `False`.
+
+Start the CPU UI only when the machine has enough memory and disk space:
+
+```bash
+docker compose -f compose.yaml -f compose.cpu.yaml up -d
+```
+
+Open <http://localhost:8001> after the health check passes. First startup may
+take a long time because it downloads and loads the model. CPU inference can be
+much slower than GPU inference, and packaging success does not prove acceptable
+model latency or that the full model fits in available RAM.
+
+Manage the CPU deployment:
+
+```bash
+# Status
+docker compose -f compose.yaml -f compose.cpu.yaml ps
+
+# Follow logs
+docker compose -f compose.yaml -f compose.cpu.yaml logs -f omnivoice
+
+# Stop services without removing them
+docker compose -f compose.yaml -f compose.cpu.yaml stop
+
+# Restart the service
+docker compose -f compose.yaml -f compose.cpu.yaml restart omnivoice
+
+# Remove the service container and network, but preserve the named cache volume
+docker compose -f compose.yaml -f compose.cpu.yaml down
+```
+
+## CUDA/NVIDIA image
+
+Build the CUDA-capable image without starting it:
+
+```bash
+docker compose --progress=plain -f compose.yaml build
+```
+
+Verify CUDA packaging without loading the model:
+
+```bash
+docker run --rm --entrypoint python omnivoice-ui:468e927 -c \
+  "import torch, gradio, omnivoice; print('Imports OK'); print('PyTorch:', torch.__version__); print('CUDA build:', torch.version.cuda); print('CUDA available:', torch.cuda.is_available())"
+```
+
+Packaging validation on a machine without an NVIDIA GPU does not prove GPU
+inference works. Validate the NVIDIA driver and container runtime on the target
+host before deployment:
+
+```bash
+nvidia-smi
+docker info
+```
+
+Build and launch on the NVIDIA host:
+
+```bash
+docker compose -f compose.yaml -f compose.gpu.yaml up --build -d
+```
+
+Manage the NVIDIA deployment:
+
+```bash
+docker compose -f compose.yaml -f compose.gpu.yaml ps
+docker compose -f compose.yaml -f compose.gpu.yaml logs -f omnivoice
+docker compose -f compose.yaml -f compose.gpu.yaml stop
+docker compose -f compose.yaml -f compose.gpu.yaml restart omnivoice
+docker compose -f compose.yaml -f compose.gpu.yaml down
+```
+
+## Persistent model cache
+
+Both deployments mount the existing named volume
+`omnivoice_huggingface-cache` at `/cache/huggingface`. Model files survive
+container stops, restarts, rebuilds, and normal `docker compose down`, avoiding
+large repeated downloads.
+
+Do not run `docker compose down -v` unless deletion of the model cache is
+explicitly intended. The `-v` option removes Compose-managed volumes, including
+the persistent Hugging Face cache.

compose.cpu.yaml

@@ -0,0 +1,17 @@
+services:
+  omnivoice:
+    build:
+      dockerfile: Dockerfile.cpu
+    image: omnivoice-ui:cpu
+    command:
+      - omnivoice-demo
+      - --model
+      - k2-fsa/OmniVoice
+      - --ip
+      - 0.0.0.0
+      - --port
+      - "8001"
+      - --device
+      - cpu
+      - --no-asr
+

compose.gpu.yaml

@@ -0,0 +1,10 @@
+services:
+  omnivoice:
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: 1
+              capabilities:
+                - gpu

compose.yaml

@@ -0,0 +1,29 @@
+services:
+  omnivoice:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    image: omnivoice-ui:468e927
+    container_name: omnivoice-ui
+    ports:
+      - "8001:8001"
+    environment:
+      HF_HOME: /cache/huggingface
+      GRADIO_ANALYTICS_ENABLED: "false"
+    volumes:
+      - huggingface-cache:/cache/huggingface
+    shm_size: "2gb"
+    healthcheck:
+      test:
+        - CMD
+        - python
+        - -c
+        - "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8001/', timeout=5)"
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 15m
+
+volumes:
+  huggingface-cache:
+    name: omnivoice_huggingface-cache

omnivoice/cli/demo.py

@@ -514,10 +514,11 @@ def main(argv=None) -> int:
         parser.print_help()
         return 0
     logging.info(f"Loading model from {checkpoint}, device={device} ...")
+    dtype = torch.float32 if str(device).startswith("cpu") else torch.float16
     model = OmniVoice.from_pretrained(
         checkpoint,
         device_map=device,
-        dtype=torch.float16,
+        dtype=dtype,
         load_asr=not args.no_asr,
         asr_model_name=args.asr_model,
     )