Can Cosmos-Reason1 model be quantized to NVFP4 format?

[2jungg]

Issue Type

• Bug Report
• Feature Request
• Question
• Documentation
• Security
• Other

Description

Steps to Reproduce (for bugs)

1. Quantize cosmos-reason1 to NVFP4 format
2. use quantized model to inference using vllm
3. Error occured "ValueError: Unsupported model when in features size is not multiple of 16"

Expected Behavior

To resolve this, I should inject my custom code to qwen2_5_vl.py to set like below

self.visual = Qwen2_5_VisionTransformer(
                config.vision_config,
                norm_eps=getattr(config, "rms_norm_eps", 1e-6),
                quant_config=None,                      # changed part
                prefix=maybe_prefix(prefix, "visual"),
                use_data_parallel=self.use_data_parallel,
            )

It makes the model well inference without any errors.
However, I want to know is there any other solutions to perfectly quantize the cosmos-reason1 model.

Environment

• Model Name/Version: cosmos-reason1-7B
• Framework (e.g., Docker, Conda): Docker image based on nvcr.io/nvidia/tritonserver:25.11-vllm-python-py3
• Python Version: python 3.12
• OS/Platform: Ubuntu
• GPU/CPU: dgx-spark (GB10)
• Additional Dependencies:

Additional Context

Deliverable