Observed
Transcribing a ~30 min audio file with VibeVoice-ASR-8bit produces a diarized transcript that ends mid-word inside a segment at ~20.5 min. Whisper transcribes the same file completely. Raising --max-tokens has no effect.
Root cause (two layers)
- mlxk:
AudioRunner.transcribe() receives the CLI's --max-tokens (run.py:563), but _transcribe_single() never forwards it to generate_transcription() — it is silently dropped. The # Ignored (Whisper generates full transcription) assumption holds for Whisper (chunked decoding, per-chunk generation, no global budget) but not for single-pass models like VibeVoice.
- mlx-audio: VibeVoice
generate() defaults to max_tokens=8192 while the same module accepts (and trims to) audio up to 59 minutes — the default budget covers only ~20 min of transcript. Internally inconsistent upstream, but out of mlxk's hands.
Constraint analysis
- The model metadata declares no output-token budget: no
generation_config.json; config.json carries only the decoder context window (decoder_config.max_position_embeddings = 131072).
- mlx-audio exposes no queryable per-model maximum —
generate_transcription() is a **kwargs passthrough with signature filtering; the 8192 is a function-signature default, not a limit.
Fix design
A free, unclamped max_tokens is not robust: an oversized budget combined with a degenerate no-EOS loop runs for hours at 100 % GPU (pseudo-hang). Therefore:
- Default: keep the upstream signature default (8192) — mlxk does not invent its own number.
- User override: forward
--max-tokens in _transcribe_single() (gen_kwargs["max_tokens"] = max_tokens), clamped to the metadata ceiling (context window from config.json). Input-aware clamping (context minus audio prefill) can only happen upstream, since the audio token count is unknown before encoding — mlxk clamps statically as a guardrail.
- mlx-audio's signature filter drops the kwarg for models whose
generate() doesn't accept it; Whisper neutrality to be verified before landing.
Acceptance
mlxk run VibeVoice-ASR-8bit --audio <30min.mp3> --max-tokens 32768 → complete transcript.
- Whisper path behavior unchanged (chunked decoding,
--translate long-form mitigation intact).
--max-tokens values above the model's context window are clamped, not forwarded verbatim.
Observed
Transcribing a ~30 min audio file with
VibeVoice-ASR-8bitproduces a diarized transcript that ends mid-word inside a segment at ~20.5 min. Whisper transcribes the same file completely. Raising--max-tokenshas no effect.Root cause (two layers)
AudioRunner.transcribe()receives the CLI's--max-tokens(run.py:563), but_transcribe_single()never forwards it togenerate_transcription()— it is silently dropped. The# Ignored (Whisper generates full transcription)assumption holds for Whisper (chunked decoding, per-chunk generation, no global budget) but not for single-pass models like VibeVoice.generate()defaults tomax_tokens=8192while the same module accepts (and trims to) audio up to 59 minutes — the default budget covers only ~20 min of transcript. Internally inconsistent upstream, but out of mlxk's hands.Constraint analysis
generation_config.json;config.jsoncarries only the decoder context window (decoder_config.max_position_embeddings = 131072).generate_transcription()is a**kwargspassthrough with signature filtering; the 8192 is a function-signature default, not a limit.Fix design
A free, unclamped
max_tokensis not robust: an oversized budget combined with a degenerate no-EOS loop runs for hours at 100 % GPU (pseudo-hang). Therefore:--max-tokensin_transcribe_single()(gen_kwargs["max_tokens"] = max_tokens), clamped to the metadata ceiling (context window fromconfig.json). Input-aware clamping (context minus audio prefill) can only happen upstream, since the audio token count is unknown before encoding — mlxk clamps statically as a guardrail.generate()doesn't accept it; Whisper neutrality to be verified before landing.Acceptance
mlxk run VibeVoice-ASR-8bit --audio <30min.mp3> --max-tokens 32768→ complete transcript.--translatelong-form mitigation intact).--max-tokensvalues above the model's context window are clamped, not forwarded verbatim.