feat: Nvim can detect venv python via "pynvim-python" tool

[drmikehenry]

Problem: Detection of the pynvim module is currently done by finding the first Python interpreter in the PATH and checking if it can import pynvim. This has several effects:

• Activation of an unrelated Python virtual environment will break automatic detection, unless pynvim is also installed in that environment.

• Installing pynvim to the expected location is difficult. User installation into the system-wide or user-wide Python site area is now deprecated. On Ubuntu 24.04 with Python 3.12, for example, the command pip install --user pynvim now fails with the error message error: externally-managed-environment.

• Users may create a dedicated virtual environment in which to install pynvim, but Neovim won't detect it; instead, they must either activate it before launching Neovim (which interferes with the user of other virtual environments) or else hard-code the variable g:python3_host_prog in their init.vim to the path of the correct Python interpreter. Neither option is desirable.

Solution: Expose pynvim's Python interpreter on the PATH under the name pynvim-python. In the typical flow:

• User installs either uv or pipx.

• User installs pynvim via:

  uv tool install --upgrade pynvim
  # Or:
  pipx install --upgrade pynvim
  

With corresponding changes in Neovim (neovim/neovim#35273), the above is all that's needed for Neovim to detect the installed location of pynvim, even if an unrelated Python virtual environments is activated. It uses standard Python tooling to automate the necessary creation of a Python virtual environment for pyenv and the publication of pynvim-python to a directory on PATH.

I've aimed to update all relevant portions of the documentation. I'm happy to make adjustments, or you may feel free to make changes as needed.

Close #593

[justinmk]

Are any of the CI failures related?

[drmikehenry]

I've added docstrings to pynvim/python.py, updated the commit message to avoid starting with a capital letter, and squashed these changes into the branch.

Tests and QA are passing on my local machine (invoked as uv tool run tox run -e py311,checkqa). This uses Neovim v0.11.3 (my installed version).

Testing against the pynvim-python branch of Neovim has one unrelated failure; the test also fails against Neovim master. I'm testing by activating the pynvim virtual environment and running:

VIMRUNTIME=/home/mike/projects/neovim/runtime PATH="/home/mike/projects/neovim/build/bin:$PATH" python3 -m pytest

The failing test is in test/test_window.py, function test_options(). The test is intended to demonstrate that a local value may be set for a global-local option without changing the global value. The test code is:

def test_options(vim: Nvim) -> None:
    vim.current.window.options['colorcolumn'] = '4,3'
    assert vim.current.window.options['colorcolumn'] == '4,3'
    # global-local option
    vim.current.window.options['statusline'] = 'window-status'
    assert vim.current.window.options['statusline'] == 'window-status'
    assert vim.options['statusline'] == ''

    with pytest.raises(KeyError) as excinfo:
        vim.current.window.options['doesnotexist']
    assert excinfo.value.args == ("Invalid option name: 'doesnotexist'",)

This chooses statusline as handy option to test, as it historically has had a known-empty default value and allows for setting an arbitrary test value.

I used git bisect to locate the change in Neovim that led to this failing test:

commit 28e31f5d3d16fac349d5e2b55837afddb822b0f3
Author: Shadman <[email protected]>
Date:   Tue Apr 22 04:05:34 2025 +0600

    feat(options): default statusline expression #33036

    Problem:
    Default 'statusline' is implemented in C and not representable as
    a statusline expression. This makes it hard for user configs/plugins to
    extend it.

    Solution:
    - Change the default 'statusline' slightly to a statusline expression.
    - Remove the C implementation.

The statusline option is no longer empty by default. The below patch (not part of this pull request) reads the default global option value before making a change to the local option, then verifies that the global option value is unchanged. With this patch, the test passes on Neovim nightly as well as stable.

diff --git a/test/test_window.py b/test/test_window.py
index 7a36d9e..9267a14 100644
--- a/test/test_window.py
+++ b/test/test_window.py
@@ -61,10 +61,11 @@ def test_vars(vim: Nvim) -> None:
 def test_options(vim: Nvim) -> None:
     vim.current.window.options['colorcolumn'] = '4,3'
     assert vim.current.window.options['colorcolumn'] == '4,3'
+    old_global_statusline = vim.options['statusline']
     # global-local option
     vim.current.window.options['statusline'] = 'window-status'
     assert vim.current.window.options['statusline'] == 'window-status'
-    assert vim.options['statusline'] == ''
+    assert vim.options['statusline'] == old_global_statusline

     with pytest.raises(KeyError) as excinfo:
         vim.current.window.options['doesnotexist']

[justinmk]

The below patch (not part of this pull request) reads the default global option value before making a change to the local option, then verifies that the global option value is unchanged. With this patch, the test passes on Neovim nightly as well as stable.

Please add that as a separate commit here, so we can get a better idea of CI status.

[drmikehenry]

I've added a commit to fix test_options().

[justinmk]

The failing test_connect_stdio test was also failing on master, though the logs are no longer available.