Skip to content

Conversation

@yiliu30
Copy link
Contributor

@yiliu30 yiliu30 commented Jan 6, 2026

As part of vllm-project/vllm#31716, the INCConfig in vllm-project/vllm will be used for GPU/CPU. However, we still need a placeholder for the INC path in the plugin.

cc @hshen14 @thuang6 @kzawora-intel @xuechendi

Signed-off-by: yiliu30 <[email protected]>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the HPU-specific INCConfig implementation to maintain compatibility as the main vLLM repository moves INC configuration for GPU/CPU use. A placeholder _FakeINCConfig is introduced to handle the "inc" quantization method in the HPU plugin while delegating actual quantization to unquantized methods.

Key Changes:

  • Introduces a new _FakeINCConfig class that acts as a stub for Intel Neural Compressor quantization
  • Monkey-patches vLLM's get_quantization_config function to intercept "inc" requests and return the fake config

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
vllm_gaudi/extension/quant.py New file defining _FakeINCConfig stub that returns unquantized methods for linear and MoE layers
vllm_gaudi/extension/ops.py Adds monkey-patch function oot_get_quantization_config to override vLLM's quantization config retrieval for "inc" method

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@yiliu30 yiliu30 changed the title Migrate HPU INCConfig Migrate INCConfig for HPU Jan 6, 2026
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
@github-actions
Copy link

github-actions bot commented Jan 6, 2026

✅ CI Passed

All checks passed successfully against the following vllm commit:
b3a2bdf1ac90748d58bf8c05f8d0095ede5c7eca

@xuechendi xuechendi self-assigned this Jan 7, 2026
@github-actions
Copy link

github-actions bot commented Jan 8, 2026

✅ CI Passed

All checks passed successfully against the following vllm commit:
0d7667419f738e44ce7f9bf311987e15b01970b0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants