[CI/Build] Replace vllm.entrypoints.openai.api_server entrypoint with vllm serve command
#25967
Conversation
Signed-off-by: DarkLight1337 <[email protected]>
DarkLight1337
requested review from
jikunshang and
bigPYJ1151
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the Dockerfiles to use the vllm serve command as the entrypoint, replacing the direct call to the python module vllm.entrypoints.openai.api_server. This change aligns the Docker images with the recommended command-line interface, improving consistency and user experience. The changes are applied correctly and consistently across the CUDA, CPU, and XPU Dockerfiles. The new entrypoint is backward compatible with existing argument passing conventions. The changes look good and I don't see any issues.
Signed-off-by: DarkLight1337 <[email protected]>
github-actions
bot
added
the
ready
|
For the Docker to be drop in replacement, we need the old flow to work as well |
|
Oh, I forgot about |
Signed-off-by: DarkLight1337 <[email protected]>
|
I have updated the CLI parsing logic to deprecate instead of erroring out when |
DarkLight1337
changed the title
vllm serve commandvllm.entrypoints.openai.api_server entrypoint with vllm serve command
Signed-off-by: DarkLight1337 <[email protected]>
mergify
bot
added
documentation
Signed-off-by: DarkLight1337 <[email protected]>
|
@simon-mo can you check again? Mamba kernels test failure is happening on main as well so it can be ignored. |
| "With `vllm serve`, you should provide the model as a " | ||
| "positional argument or in a config file instead of via " | ||
| "the `--model` option.") | ||
| "the `--model` option. " | ||
| "The `--model` option will be removed in v0.13.") |
There was a problem hiding this comment.
Is there any harm in allowing both indefinitely?
There was a problem hiding this comment.
I am not sure who removed the ability to use --model in the first place. Maybe @mgoin ?
Signed-off-by: DarkLight1337 <[email protected]>
…th `vllm serve` command (vllm-project#25967) Signed-off-by: DarkLight1337 <[email protected]>
…th `vllm serve` command (#25967) Signed-off-by: DarkLight1337 <[email protected]> Signed-off-by: yewentao256 <[email protected]>
Purpose
vllm serve <model>works now, without having to add--model. So I think it's safe to just change the command directly.Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.