Question: how to use with vLLM?

[test3211234]

When trying to pip install vllm on Windows I get an error about vllm doesn't work on ROCm and Windows. So I think I can use WSL, which I don't know how, and also I don't know about the distinction between ZLUDA and ROCm and how to use ZLUDA.

[lshqqytiger]

You'll install vllm package for CUDA 11.8. Also, if needed, replace dll files inside vllm (or torch? I don't know)
However, if vLLM depends on libraries that is not supported/implemented such as cuDNN and it does not allow to disable them, it won't work.