Error occurs occasionally when shutdown uvicorn(using Docker and multiple workers)

[tim-chow]

My test code shown below:

% ls
Dockerfile test.py

# test.py

import uvicorn
from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def root():
    return {"hello": "world"}


if __name__ == "__main__":
    uvicorn.run("test:app", host="0.0.0.0", port=8000, workers=10)

# Dockerfile

FROM python:3.10.7-buster
COPY test.py /
RUN pip install fastapi uvicorn
CMD python /test.py

And then build an image in my work directory:

docker build -t test-uvicorn:20220916 .

Start docker container using the command shown below:

docker run --rm -it  test-uvicorn:20220916

Wait for a moment, and then press CTRL + C, Maybe get outputs like below:

INFO:     Finished server process [12]
INFO:     Waiting for application shutdown.
INFO:     Waiting for application shutdown.
INFO:     Waiting for application shutdown.
INFO:     Waiting for application shutdown.
ERROR:    Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 648, in lifespan
    await receive()
  File "/usr/local/lib/python3.10/site-packages/uvicorn/lifespan/on.py", line 137, in receive
    return await self.receive_queue.get()
  File "/usr/local/lib/python3.10/asyncio/queues.py", line 159, in get
    await getter
asyncio.exceptions.CancelledError

when I repeat these operations many times, The above error will occur occasionally.

[euri10]

does it happen without docker ?

[tim-chow]

@euri10 It looks all right without docker.

[Kludex]

You mean that you are doing CTRL + C multiple times? If that's the case, then it's normal...

[tim-chow]

@Kludex I mean that I repeat these operations - [docker run ..., wait for a moment, ..., press CTRL + C] many times. rather than repeating these opertions - docker run, ..., wait for a moment, then only press CTRL + C many times.

[autra]

I reproduce the same error every time, with or without docker, but only with --workers=1, with the following code:

from multiprocessing import Process
from fastapi import FastAPI
from contextlib import asynccontextmanager
import time


class MyP(Process):

    def __init__(self):
        super().__init__()

    def run(self):
        while True:
            print('foo')
            time.sleep(1)
        print('finished')


@asynccontextmanager
async def lifespan(app):
    myp = MyP()
    myp.start()
    from asyncio.exceptions import CancelledError
    try:
        yield
    except CancelledError:
        print('Got CancelledError, exiting')
    myp.terminate()
    myp.join()


app = FastAPI(
    title="demo that hangs when you ctrl+c",
    description="""
    """,
    lifespan=lifespan,
)

@app.get("/")
def hello_world():
    return {"hello": "world!"}


if __name__ == '__main__':
    myp = MyP()
    myp.start()
    time.sleep(5)
    myp.terminate()
    myp.join()

When executing directly with python3 ./file.py, it terminates normally.

When executing with uvicorn --workers=2 file:app and I hit ctrl+c, uvicorn shutdowns correctly.

When executing with uvicorn --workers=1 file:app and I hit ctrl+c, uvicorn doesn't shut down and foo keeps being printed. The only way to shut it is to use kill -9 on the process.

Hope this helps.

[autra]

More details:

os : ubuntu 22.04

ᐅ python --version
Python 3.9.16
ᐅ uvicorn --version
Running uvicorn 0.22.0 with CPython 3.9.16 on Linux
ᐅ pip list | grep -i fastapi
fastapi                       0.97.0

[autra]

And last detail, I don't reproduce with --reload either.

For the 2 cases where it stops correctly, I actually have a KeyboardInterrupt on the time.sleepfunction. I don't have that one with--workers=1`

[zanieb]

@autra I don't think you can just swallow the CancelledError like that — it's important that it is re-raised for cancellation to occur. Do you reproduce a hang if you add a raise there?