[Errno 98] error while attempting to bind on address, but only one process is running

[youkaichao]

Hi, team, thanks for the great framework!

I'm a contributor for the vLLM project, and we use uvicorn for the request handling:

https://github.com/vllm-project/vllm/blob/fc5ebbd1d3453461ea6e00a78faf87c41d1aa625/vllm/entrypoints/launcher.py#L6

Recently, I met a strange bug happening in our ci, saying that the port we use for uvicorn is already used. The stack trace looks like:

[2024-08-21T22:03:06Z] INFO:     Started server process [21410]
[2024-08-21T22:03:06Z] INFO:     Waiting for application startup.
[2024-08-21T22:03:06Z] INFO:     Application startup complete.
[2024-08-21T22:03:06Z] ERROR:    [Errno 98] error while attempting to bind on address (‘0.0.0.0’, 37017): address already in use
[2024-08-21T22:03:06Z] INFO:     Waiting for application shutdown.
[2024-08-21T22:03:06Z] INFO:     Application shutdown complete.
[2024-08-21T22:03:06Z] DEBUG 08-21 22:03:06 launcher.py:64] port 37017 is used by process psutil.Process(pid=21410, name=‘pt_main_thread’, status=‘running’, started=‘22:00:32’) launched with command:
[2024-08-21T22:03:06Z] DEBUG 08-21 22:03:06 launcher.py:64] /usr/bin/python3 /usr/local/bin/vllm serve JackFram/llama-160m --disable-log-requests --use-v2-block-manager --worker-use-ray --gpu-memory-utilization 0.85 --swap-space 16 --enforce-eager --tensor-parallel-size 2 --pipeline-parallel-size 2 --port 37017

This has been bothering us for a long time. And then I added quite a lot logging information to find out which process is using that port.

Surprisingly, I find it is the process itself, that uses the port. You can see the server process has pid 21410. Then it fails to bind the port 37017. Then the log shows port 37017 is used by process with pid=21410.

Is this some potential bug from uvicorn? Do you have any ideas to avoid it?

Thanks!