Skip to content

feat: add Apple Silicon Qwen3-TTS MLX backends - #8

Open
groxaxo wants to merge 1 commit into
mainfrom
feat/apple-silicon-qwen3-tts-mlx
Open

feat: add Apple Silicon Qwen3-TTS MLX backends#8
groxaxo wants to merge 1 commit into
mainfrom
feat/apple-silicon-qwen3-tts-mlx

Conversation

@groxaxo

@groxaxo groxaxo commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a native Apple Silicon serving path while preserving the existing VibeVoice CUDA/realtime workflow.

  • serves the exact aufklarer/Qwen3-TTS-12Hz-1.7B-Base-MLX-8bit bundle through Speech Swift
  • serves mlx-community/Qwen3-TTS-12Hz-1.7B-Base-4bit through the model-compatible mlx-audio==0.3.0 loader
  • adds an adapter-driven FastAPI application for native/non-realtime models
  • adds an arm64 macOS bootstrap, model aliases, reference-audio conditioning, health/config/model endpoints, and OpenAI-style speech output
  • keeps the vendored realtime VibeVoice application isolated so it cannot advertise or route requests to a Qwen adapter it never loaded

Inconsistencies fixed

  1. macOS was documented, but the active registry and launcher only supported PyTorch VibeVoice profiles.
  2. scripts/run_server.py exited instead of serving non-realtime adapters.
  3. Model sources were coerced to local paths, breaking Hugging Face ID fallback on a fresh Mac.
  4. Registry/launcher imports eagerly pulled PyTorch backends into a Qwen-only Mac process.
  5. The old realtime app could resolve model names while still synthesizing with one fixed realtime engine.
  6. An empty/uninitialized VibeVoice submodule could pass the old bootstrap directory check.
  7. The VibeVoice and MLX dependency stacks require incompatible Transformers and Hugging Face Hub generations; they are now isolated as mutually exclusive uv extras.
  8. mlx-audio expects reference audio as a loaded/resampled MLX waveform, not a filesystem string; the adapter now performs that conversion explicitly.

Architecture

  • realtime-0.5b: unchanged subprocess-based VibeVoice demo, including its web UI and streaming route.
  • qwen3-tts-mlx-8bit: Speech Swift CLI, using the exact requested model ID.
  • qwen3-tts-mlx-4bit: in-process mlx-audio adapter with lazy model loading and serialized MLX generation.
  • one active model per process; a request selecting another model returns HTTP 409 instead of silently using the wrong backend.

Local validation

  • python -m pytest -q test_macos_mlx.py17 passed
  • python -m compileall -q runner scripts main.py test_macos_mlx.py
  • TOML parsing and dependency-stack assertions
  • AST parsing and duplicate top-level definition scan across the patched Python files
  • bash -n scripts/bootstrap_uv.sh scripts/bootstrap_macos.sh
  • compare check: one commit ahead of main, exactly 15 intended files, no accidental backend stubs

No GitHub Actions workflow was added or used.

Remaining target-hardware check

A real model download and Metal/Speech Swift synthesis smoke test must be run on native Apple Silicon macOS; the implementation environment used for this patch is Linux and cannot execute MLX/Metal or Speech Swift.

- add exact aufklarer 1.7B 8-bit support through Speech Swift
- add 1.7B 4-bit support through mlx-audio
- introduce adapter-driven native FastAPI serving and model isolation
- preserve the existing VibeVoice realtime CUDA subprocess path
- isolate incompatible Transformers and Hugging Face Hub dependency stacks
- add native macOS bootstrap, documentation, and hardware-free integration tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant