Poetry with WatchFiles - leads to a bug

[zvolsky]

• Poetry version: 1.4.0
• Python version: 3.11.2
• OS version and name: Debian 11 stable
• pyproject.toml:

[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.94.1"
uvicorn = {extras = ["standard"], version = "^0.21.0"}
watchfiles = "^0.18.1"

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Same is mentioned here (the "similar problem" mentioned later on the page).

WatchFiles is used as observer of file changes for FastApi+Uvicorn.
When I make the virtual environment using Poetry (and install fastapi + uvicorn[standard]), then uvicorn --reload is not able to run and fails.
When I make "same" virtual environment using python -m venv and pip, everything works well.

I don't say it is an bug of poetry, however it would be good to investigate the problem more: install the "Hello world" example for FastApi+uvicorn and think a little why the internal Rust watcher fails - just in case when the virtual environment is created using poetry.

I don't believe that this is really "too many open files" problem. Because we are here with the "Hello world" application only. So probably the Rust watcher fails from another reason (when poetry handles the environment) and WachFiles reports bad message.

Traceback (most recent call last):
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/bin/uvicorn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/uvicorn/main.py", line 403, in main
    run(
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/uvicorn/main.py", line 563, in run
    ChangeReload(config, target=server.run, sockets=[sock]).run()
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/uvicorn/supervisors/basereload.py", line 45, in run
    for changes in self:
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/uvicorn/supervisors/basereload.py", line 64, in __next__
    return self.should_restart()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/uvicorn/supervisors/watchfilesreload.py", line 85, in should_restart
    changes = next(self.watcher)
              ^^^^^^^^^^^^^^^^^^
  File "/home/mirek/py/fa/eventbook/eventbook/.venv/lib/python3.11/site-packages/watchfiles/main.py", line 120, in watch
    with RustNotify([str(p) for p in paths], debug, force_polling, poll_delay_ms, recursive) as watcher:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_rust_notify.WatchfilesRustInternalError: Error creating recommended watcher: Too many open files (os error 24)

[samuelcolvin]

Most likely explanation is that poetry is installing the wrong binary.

Please can you check which file is downloaded from pypi and installed with venv and which is installed with poetry.

[samuelcolvin]

As a work around you should be able to set the WATCHFILES_FORCE_POLLING environment variable, but it would still be good to understand what poetry is doing differently to cause this problem.

[dimbleby]

feels like you've raised this too soon. If and when you discover a poetry bug, let us know!

[zvolsky]

Hallo Samuel (& dimbleby),

I am very sorry for this.
I was going to try WATCHFILE_FORCE_POLLING env and find the required info, however after switching to the poetry installation I see that it works today(!) well.
So I remade the whole project + venv once again using poetry ... and it still works.
So it is my mistake that I haven't restarted (or reinstalled?) 3-times before writing something to Issues.
But I have had tendency to do this mistake, because I read about exactly same problem of user DazEdword in the uvicorn's issue. - So there is probably something not 100% correct. But I am not able to reproduce it for now.

Once again sorry for this.
And I hope that tommorow the error will not occur again :)

[dimbleby]

please close

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.