Add MLX / Apple Silicon (MPS) backend support#345
Open
walkoncross wants to merge 4 commits into
Open
Conversation
- 新增 backend_utils 统一解析设备/dtype/attn：mlx 与 auto 自动映射到 本机可用后端（Apple Silicon 落到 mps，无则 cpu） - mps 上 bfloat16 自动降级为 float16，非 CUDA 下自动关闭 flash_attention_2 - from_pretrained(模型/tokenizer) 与 demo CLI 走统一解析，默认设备自动探测 - examples 去除写死的 cuda:0 与 torch.cuda.synchronize，改用跨后端同步 - gitignore 忽略本地模型目录产出的音频 验收：examples/test_model_12hz_base.py 在 mps 上 12 个 case 全部跑通
上次批量替换漏掉了 batch 段（后跟空行而非 t1）的 torch.cuda.synchronize， 在无 CUDA 的 mps 环境会抛 AssertionError。改为跨后端的 synchronize_device。 验收：四个 example 脚本在 mps 上均已实测通过（rc=0）
- 四个 example 脚本的音频输出统一到 outputs/<脚本名>/,目录名由 __file__ 自动派生,脚本与输出目录一一对应 - 之前 base 用独立长目录名、其余三个直接写仓库根目录,不统一 - gitignore 旧输出目录规则替换为 outputs/ 验收:四个脚本在 mps 上均 rc=0,产物落到各自 outputs 子目录 (base 20 / custom 3 / design 3 / tokenizer 9)
- README 手动下载命令原本存到裸 ./<模型名>/,与 examples/finetuning 脚本里的 Qwen/<模型名> 加载路径对不上,照文档下载后需手动改路径 - 12 条 modelscope/huggingface-cli 命令统一改为 ./Qwen/<模型名>/ - 补充说明:从仓库根目录运行时脚本可直接读到本地权重,无需改路径 - 脚本保持 Qwen/<模型名> repo id 双关写法,不影响缺失时自动下载
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & Why
Enables running Qwen3-TTS on Apple Silicon (and CPU) out of the box. Previously the examples and CLI hard-coded
cuda:0+flash_attention_2, so they could not run on machines without CUDA.Changes
qwen_tts/inference/backend_utils.py(new) — unified device/dtype/attn resolver:default_device()auto-detectscuda→ Apple Siliconmps→cpumlx/auto/mpsaliases route to the best locally available backendbfloat16auto-falls back tofloat16on MPS;flash_attention_2auto-disabled off-CUDAsynchronize_device()wraps CUDA/MPS syncfrom_pretrained(model + tokenizer) andcli/demo.pyroute through the resolver; CLI--devicenow auto-detects.examples/*.py— drop hard-codedcuda:0/torch.cuda.synchronize(); unify audio output underoutputs/<script-name>/.--local_dircommands now keep theQwen/prefix so the downloaded layout matches the defaultQwen/<model-name>paths in the scripts.Scripts keep the
Qwen/<model-name>repo-id form, sofrom_pretrainedauto-download on cache miss is preserved.Verification
Ran on Apple Silicon (torch 2.12, MPS, no CUDA) — all four example scripts exit 0:
test_tokenizer_12hz.py,test_model_12hz_base.py(12/12 cases),test_model_12hz_custom_voice.py,test_model_12hz_voice_design.pyNot tested on CUDA hardware — the CUDA path is unchanged by design (resolver is a no-op when CUDA is available), but a maintainer sanity-check on a GPU box is welcome.