Add Dockerized CPU deployment and Gradio UI configuration#232
Add Dockerized CPU deployment and Gradio UI configuration#232khanhhuyenpham wants to merge 2 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces Docker deployment support for OmniVoice, adding Dockerfiles, Docker Compose configurations for both CPU and GPU environments, and a detailed README guide. Additionally, the CLI demo script was updated to dynamically set the PyTorch data type based on the device. The feedback suggests restricting torch.float16 to cuda and xpu devices to avoid compatibility issues on Apple Silicon (MPS), and replacing the hardcoded commit hash image tag with a generic tag like latest in the Docker Compose and documentation files.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| parser.print_help() | ||
| return 0 | ||
| logging.info(f"Loading model from {checkpoint}, device={device} ...") | ||
| dtype = torch.float32 if str(device).startswith("cpu") else torch.float16 |
There was a problem hiding this comment.
Using torch.float16 on Apple Silicon (MPS) can lead to runtime errors or numerical instability due to incomplete operator support for half-precision on MPS. To ensure compatibility and consistency with the ASR model's dtype selection (which uses float32 for non-CUDA/non-XPU devices), we should use torch.float16 only for cuda and xpu devices, and default to torch.float32 for others (including cpu and mps).
| dtype = torch.float32 if str(device).startswith("cpu") else torch.float16 | |
| dtype = torch.float16 if str(device).startswith(("cuda", "xpu")) else torch.float32 |
| build: | ||
| context: . | ||
| dockerfile: Dockerfile | ||
| image: omnivoice-ui:468e927 |
There was a problem hiding this comment.
The default image tag is hardcoded to a specific commit hash (468e927). This can cause issues if users try to run the compose file without building it locally, or if they expect a standard tag like latest or gpu to match the cpu tag in compose.cpu.yaml. It is recommended to use a generic tag like latest or gpu for the default image.
image: omnivoice-ui:latest| Verify CUDA packaging without loading the model: | ||
|
|
||
| ```bash | ||
| docker run --rm --entrypoint python omnivoice-ui:468e927 -c \ |
There was a problem hiding this comment.
The smoke test command uses the hardcoded commit hash tag omnivoice-ui:468e927. This should be updated to use the generic latest or gpu tag to match the default compose configuration.
| docker run --rm --entrypoint python omnivoice-ui:468e927 -c \ | |
| docker run --rm --entrypoint python omnivoice-ui:latest -c \ |
2 participants
No description provided.