Skip to content

fix(api): allow nested dicts in ChatMessage.content for multimodal requests - #125

Open
creedx12 wants to merge 1 commit into
madroidmaq:mainfrom
creedx12:fix/chatmessage-content-multimodal-type
Open

creedx12 wants to merge 1 commit into
madroidmaq:mainfrom
creedx12:fix/chatmessage-content-multimodal-type

Conversation

@creedx12

@creedx12 creedx12 commented Jun 15, 2026

Copy link
Copy Markdown

Problem

ChatMessage.content is typed Optional[Union[str, List[Dict[str, str]]]] (chat/openai/schema.py). The OpenAI multimodal format nests a dict under image_url:

{"type": "image_url", "image_url": {"url": "..."}}

Dict[str, str] rejects that nested object, so FastAPI returns 422 before the handler runs — every vision request fails at the schema layer, even though VLM routing exists internally.

Fix

Widen the value type to Any (List[Dict[str, Any]]). Any is already imported. One-line change.

Present on main (0.5.3).

Summary by CodeRabbit

  • Improvements
    • Chat messages now support more flexible content structures and richer data formats for enhanced compatibility.

ChatMessage.content was typed Optional[Union[str, List[Dict[str, str]]]].
The OpenAI multimodal format nests a dict under image_url:
  {"type": "image_url", "image_url": {"url": "..."}}
so Dict[str, str] fails validation and FastAPI returns 422 before the handler
runs — every vision request is rejected at the schema layer.

Widen the value type to Any (Dict[str, Any]) so multimodal content validates.
Any is already imported.
@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6a4c0a3e-f0df-439a-8033-5ef0334621c3

📥 Commits

Reviewing files that changed from the base of the PR and between 4f8e9ef and dd2666d.

📒 Files selected for processing (1)
  • src/mlx_omni_server/chat/openai/schema.py

📝 Walkthrough

Walkthrough

The type annotation for ChatMessage.content in the OpenAI schema is widened from Optional[Union[str, List[Dict[str, str]]]] to Optional[Union[str, List[Dict[str, Any]]]], allowing dictionary values in the list form to hold any type rather than only strings.

Changes

ChatMessage Schema Type Widening

Layer / File(s) Summary
ChatMessage.content field type
src/mlx_omni_server/chat/openai/schema.py
content field type changes from List[Dict[str, str]] to List[Dict[str, Any]] for the list branch, widening Pydantic validation to accept non-string dictionary values.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Poem

A bunny once hopped through a schema one day,
And found that the strings were too strict in their way.
"Let Any come through!" cried the rabbit with glee,
One character changed — str swapped out for Any.
Now content flows freely, as typed as can be! 🐇✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: updating ChatMessage.content to accept nested dictionaries for multimodal requests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant