This repository was archived by the owner on Nov 10, 2025. It is now read-only.
This repository was archived by the owner on Nov 10, 2025. It is now read-only.
vllm in-process handles sampling temperature incorrectly #194
Description
Describe the bug
When using decoding method "sample", the temperature is not handled correctly.
- The default temperature is 0.0, instead of 1.0.
- If the user sets a temperature, the default of 0.0 will override the user value.
Platform
Please provide details about the environment you are using, including the following:
- Interpreter version: 3.12.7
- Library version: 4151c97
The bug is at vllm_in_proc.py:461