Skip to content

fix(vlm): expose _tokenizer on VLMTokenizerWrapper for guided decoding - #124

Open
creedx12 wants to merge 1 commit into
madroidmaq:mainfrom
creedx12:fix/vlm-tokenizer-_tokenizer-attr
Open

creedx12 wants to merge 1 commit into
madroidmaq:mainfrom
creedx12:fix/vlm-tokenizer-_tokenizer-attr

Conversation

@creedx12

@creedx12 creedx12 commented Jun 15, 2026

Copy link
Copy Markdown

Problem

VLMTokenizerWrapper.__init__ (chat/mlx/model_types.py) sets self.tokenizer but not self._tokenizer. OutlinesLogitsProcessor reads tokenizer._tokenizer (chat/mlx/outlines_logits_processor.py:48,58), so any request that uses json_schema / guided decoding against a VLM model raises:

AttributeError: 'VLMTokenizerWrapper' object has no attribute '_tokenizer'

Fix

Alias self._tokenizer = self.tokenizer in __init__ so the Outlines path resolves the underlying tokenizer for VLM models. One-line change (+ comment).

Present on main (0.5.3). Verified against Gemma-4 with a json_schema response format.

Summary by CodeRabbit

  • Bug Fixes
    • Fixed a compatibility error in Vision Language Model support that was preventing advanced decoding features from working properly.

VLMTokenizerWrapper.__init__ sets self.tokenizer but OutlinesLogitsProcessor
reads tokenizer._tokenizer (outlines_logits_processor.py:48,58). Any json_schema
/ guided-decoding request against a VLM therefore raises
AttributeError: 'VLMTokenizerWrapper' object has no attribute '_tokenizer'.

Alias self._tokenizer = self.tokenizer in __init__ so the Outlines path works
with VLM models.
@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9e1c836a-fc90-4d1b-a1e9-82b8aada2f16

📥 Commits

Reviewing files that changed from the base of the PR and between 4f8e9ef and 3cb8e0e.

📒 Files selected for processing (1)
  • src/mlx_omni_server/chat/mlx/model_types.py

📝 Walkthrough

Walkthrough

VLMTokenizerWrapper inside load_mlx_model gains a _tokenizer attribute assigned from self.tokenizer. This exposes the attribute expected by downstream logits processing and guided/json-schema decoding components that access tokenizer._tokenizer on VLM models.

Changes

VLM Tokenizer Compatibility Fix

Layer / File(s) Summary
Expose _tokenizer on VLMTokenizerWrapper
src/mlx_omni_server/chat/mlx/model_types.py
Adds self._tokenizer = self.tokenizer with explanatory comments inside VLMTokenizerWrapper so decoding paths that access tokenizer._tokenizer no longer raise AttributeError on VLM models.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Poem

A wrapper once bare, now patched with care,
_tokenizer assigned, no error in the air! 🐇
The VLM decodes without a fright,
Six lines of fix make everything right.
Hop hop hooray, the attribute's there! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title clearly and specifically describes the main change: exposing the _tokenizer attribute on VLMTokenizerWrapper to fix guided decoding compatibility.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant