[Misc]: Upstream of this work

[KMS07]

Anything you want to discuss about vllm.

Authors may I know where is this work integrated to the vllm

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.