Unsupported model types

[ckuethe]

Testing b1006 on strix halo (gfx1151), kernel 7.0.0-13, ROCm 7.12 (https://rocm.nightlies.amd.com/tarball/therock-dist-linux-gfx1151-7.12.0a20260311.tar.gz)

The following model types were detected raised what(): Unsupported model type:

• Dream
• bitnet
• fish_qwen3_omni
• florence2
• gemma3
• gemma4
• kimi_vl
• kitten_tts
• paddleocr_vl
• qwen2_5_vl
• qwen3_asr
• qwen3_tts
• qwen3_vl
• sam_audio
• sesame
• voxcpm2
• voxtral_tts

Models

• argmaxinc/mlx-FLUX.1-schnell
• exdysa/Flex.2-preview-MLX
• majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-8bit
• mlx-community/AceReason-Nemotron-7B-4bit
• mlx-community/Devstral-Small-2505-4bit-DWQ
• mlx-community/Dia-1.6B
• mlx-community/DiffuCoder-7B-cpGRPO-8bit
• mlx-community/Falcon3-10B-Instruct-1.58bit-4bit
• mlx-community/Florence-2-large-ft-bf16
• mlx-community/GLM-Z1-32B-0414-4bit
• mlx-community/Granite-4.0-H-Tiny-4bit-DWQ
• mlx-community/Kimi-VL-A3B-Thinking-4bit
• mlx-community/Kokoro-82M-bf16
• mlx-community/Llama-OuteTTS-1.0-1B-4bit
• mlx-community/Mellum-4b-base
• mlx-community/Nanonets-OCR-s-8bit
• mlx-community/OpenELM-3B
• mlx-community/PaddleOCR-VL-8bit
• mlx-community/QwQ-32B-4bit
• mlx-community/Qwen3-8B-4bit
• mlx-community/Qwen3-8B-4bit-DWQ-053125
• mlx-community/Qwen3-ASR-1.7B-4bit
• mlx-community/Qwen3-Next-80B-A3B-Thinking-8bit
• mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit
• mlx-community/Qwen3-VL-4B-Instruct-8bit
• mlx-community/Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-4bit
• mlx-community/SmolLM3-3B-4bit
• mlx-community/VisualQuality-R1-7B-8bit
• mlx-community/VoxCPM2-4bit
• mlx-community/Voxtral-4B-TTS-2603-mlx-6bit
• mlx-community/bitnet-b1.58-2B-4T
• mlx-community/codegemma-7b-it-4bit
• mlx-community/csm-1b
• mlx-community/demucs-mlx
• mlx-community/embeddinggemma-300m-8bit
• mlx-community/embeddinggemma-300m-qat-q8_0-unquantized-bf16
• mlx-community/fishaudio-s2-pro-8bit-mlx
• mlx-community/gemma-3-270m-it-8bit
• mlx-community/gemma-3-27b-it-qat-4bit
• mlx-community/gemma-4-26B-A4B-it-heretic-msq-2.6bit
• mlx-community/kitten-tts-mini-0.8-8bit
• mlx-community/lille-130m-instruct-8bit
• mlx-community/sam-audio-base
• txgsync/gpt-oss-120b-Derestricted-mxfp4-mlx
• uqer1244/MLX-z-image
• waltgrace/mlx-expert-sniper

[ckuethe]

The following models didn't crash. Not saying they made sense; only that they didn't crash.

• mlx-community/Qwen3-8B-4bit-DWQ-053125
• mlx-community/Qwen3-8B-4bit
• mlx-community/Qwen3-Next-80B-A3B-Thinking-8bit
• mlx-community/Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-4bit
• mlx-community/SmolLM3-3B-4bit
• mlx-community/embeddinggemma-300m-qat-q8_0-unquantized-bf16

[Geramy]

these models should be fixed now.

mlx-community/Qwen3-8B-4bit-DWQ-053125
mlx-community/Qwen3-8B-4bit
mlx-community/Qwen3-Next-80B-A3B-Thinking-8bit
mlx-community/Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-4bit```

[bong-water-water-bong]

Status update

PR #41 adds support for bitnet model type (BitNet 1.58-bit ternary inference), which was one of the model types listed in this issue.

Current model type coverage

The lemon-mlx-engine now supports 40+ LLM model types and 14 VLM model types. Here's how the model types from this issue map:

Now supported (LLM):

• bitnet — Added in PR Add BitNet 1.58-bit ternary model support #41 ✅

Already supported (LLM):

• gpt_oss, granitemoehybrid, openelm, smollm3, qwen3, qwen3_next, glm4, deepseek_v3, phi3, gemma2, mistral, llama, cohere, starcoder2, etc.

Supported as VLM (use the VLM factory, not LLM):

• gemma3 — Supported as VLM (Gemma3Configuration)
• qwen2_5_vl — Supported as VLM
• qwen3_vl — Supported as VLM
• mistral3 — Supported as both LLM and VLM

Not yet supported (require new implementations):

• Dream — Diffusion-based LLM (Dream-org/Dream-v0-Instruct-7B). Uses a fundamentally different architecture (diffusion over tokens) that would need a new model implementation.
• gemma4 / gemma4_unified — Gemma 4 unified VLM (Gemma4UnifiedForConditionalGeneration). New architecture, would need a new VLM model implementation.
• fish_qwen3_omni, qwen3_asr, qwen3_tts, voxtral_tts, sam_audio, sesame, kitten_tts, voxcpm2 — Audio/TTS/ASR models. These are non-text-generation models and are likely out of scope for lemon-mlx-engine (an LLM inference engine).
• florence2, kimi_vl, paddleocr_vl — VLM/OCR models that would need VLM implementations.
• codegemma — Uses gemma model type, should already work.

Non-model-type issues

Some models in the list fail for reasons other than unsupported model types:

• mlx-community/bitnet-b1.58-2B-4T — Now works (PR Add BitNet 1.58-bit ternary model support #41)
• mlx-community/GLM-Z1-32B-0414-4bit — Now works (PR Add BitNet 1.58-bit ternary model support #41, linear_forward fix for quantized lm_head)
• mlx-community/Granite-4.0-H-Tiny-4bit-DWQ — Loads and runs (no crash), but SSM output quality issue (issue hipMalloc (unified) failed: an illegal memory access was encountered. #6)
• mlx-community/OpenELM-3B — Now loads without segfault (PR Add BitNet 1.58-bit ternary model support #41, issue Segmentation fault near hipblaslt with OpenELM #7)
• txgsync/gpt-oss-120b-Derestricted-mxfp4-mlx — Now loads (PR Add BitNet 1.58-bit ternary model support #41, MXFP4 support, issue [gather_qmm] Biases must be provided for affine quantization. #10)