Tooling: dealing with multiple py environments

[euri10]

Raised by comment here

I'm almost only involved in uvicorn, but as this discussion deals with tooling, at @tomchristie request I put it here since I'm sure this will be more visible (maybe we could have a Encode specific discussion is possible, not sure ?)

Today on uvicorn, my process when/if I want/have to deal with a py version discrepancy (check if something works the same on py36 for instance) I have to

rm -rf venv
asdf local python 3.6.13
./scripts/install
./scripts/test

This supposes one thing obviously : to have the base interpreter on my debian machine, for that I'm using the nice asdf tool that let me pick a python interpreter that will be used locally (think of asdf as pyenv on steroids as it deals with python interpreter but can do the same for rust, java, etc, etc...)

It is entirely possible one would prefer to deal with that base interpreter dance differently, I think it's better not to mess with your distro interpreter but ymmv ! In any case this is not very relevant to the issue at end since in any case you'll need the py36 should you want to create a venv with it.

Once the interpreter is setup then the install script will take care of setting up de dev venv as you know it.

When I have to debug something on Windows, I spawn a libvirt win10 image, and also on that machine I have to have pyenv installed, if I want to test on that OS a specific python version behaviour.

Should we use tox (or nox, I'm no expert enough on the discrepancies between the 2) I'll just have to tox -e py36 and provided I have the base interpreter installed then this is all I have to do. I want to switch and test python 3.10, then I do tox -e py310

The same applies to lint / format environments : in current setup a single venv is tied to a single py interpreter and all the dev dependencies are in it.
With tox/nox one can have a seperate venv for formatting stuff, and it does not need to have all the tooling installed.

So overall it seems to me the tox/nox benefits lies in the easy switch and that it significantly eases that process.

[tomchristie]

So...

The benefits of tox and nox come primarily when you want to run the tests against multiple versions of Python. For almost everything I've done with encode I find that's actually reasonably rare. I'm almost always okay with the default case being to run the tests against a particular given version of Python, and allowing the CI to discover any cases where we've got version specific issues.

However, I do want to be able to switch between different versions explicitly, because we really do need to be able to be explicit about which python version we're running on, and to be able to switch out to different versions when needed.

The scripts/install script does currently allow you to be explicit about which Python executable to use on install, but the behaviour could be a little neater.

The most well supported tooling for managing multiple Python installs seems to be pyenv, so here's my suggestion...

An install, where no venv has been created yet...

# Note: If `pyenv` is installed, then the script will use `pyenv prefix`/bin/python when creating the venv.
# If `pyenv` is not installed, the script will default to using `python3`.
./scripts/install
./scripts/test

An install, where a venv has already be created...

# Note: Running the install script will *always* delete any existing `venv` and re-install from scratch,
# ensuring it always generates a clean environment.
./scripts/install
./scripts/test

An install, where no venv has been created yet, with a version specified...

# Note: If `pyenv` is installed, then the script will use `pyenv prefix <version>`/bin/python when creating the venv.
# If `pyenv` is not installed, the will exit with an error.
./scripts/install 3.8.7
./scripts/test

An install, where a venv has already be created, with a version specified...

# Note: Switching to testing against a different Python version is nice and easy here.
./scripts/install 3.10.2
./scripts/test

Managing installed Python versions...

# We wouldn't do anything in this space other than rely on the existing `pyenv` functionality...
pyenv versions
pyenv install
pyenv uninstall
pyenv global

My overriding aim here is to keep the tooling absolutely as lightweight as possible. I think making the tweaks suggested here would meet the aim of being able to explicitly deal with multiple python environments cleanly, without introducing any significant extra toolchains.

[Kludex]

The benefits of tox and nox come primarily when you want to run the tests against multiple versions of Python. For almost everything I've done with encode I find that's actually reasonably rare. I'm almost always okay with the default case being to run the tests against a particular given version of Python, and allowing the CI to discover any cases where we've got version specific issues.

I have a different perception, I guess mostly because of the mypy compliance on uvicorn, and the different markers available there. I lost a lot of time going back and forward on the CI to get environment related errors, that could have been solved by running a single tox instance.

Btw, I use pyenv.