pip_parse builds wheels with different names

[linzhp]

🐞 bug report

Affected Rule

pip_parse

Is this a regression?

No

Description

When building PyPika 0.48.9, setuptools 75.8.0 creates a wheel called PyPika-0.48.9-py2.py3-none-any.whl, while 75.8.1 creates pypika-0.48.9-py2.py3-none-any.whl (pypa/setuptools#4766). When isolated = True, pip_parse relies on pip to determine which version of setuptools to build wheels of PyPika from source. At some point, pip decided to switch from 75.8.0 to 75.8.1. This caused some machines having PyPika-0.48.9-py2.py3-none-any.whl while some others having pypika-0.48.9-py2.py3-none-any.whl, leading to different checksums and cache keys during Bazel query/build.

Can we force pip to use the version of setuptools managed by Bazel?

🔬 Minimal Reproduction

By installing specific setuptools version and run pip wheel in non-isolated mode, we can see the different wheel names created:

(myenv) ➜  myenv pip install setuptools==75.8.0
...
Successfully installed setuptools-75.8.0
(myenv) ➜  myenv pip wheel --no-build-isolation pypika
...
Building wheels for collected packages: pypika
  Building wheel for pypika (PEP 517) ... done
  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53771 sha256=976845cf855a2cff476a284ac81f0824ce3ba8bd69c5090d07fc302c0edcd042
  Stored in directory: /home/user/.cache/pip/wheels/3b/59/32/380c4d773968f655e18971a44e5a99f96dc2b9fb76287dae59
Successfully built pypika
(myenv) ➜  myenv rm -r /home/user/.cache/pip/wheels/3b/59/32/380c4d773968f655e18971a44e5a99f96dc2b9fb76287dae59 
(myenv) ➜  myenv rm PyPika-0.48.9-py2.py3-none-any.whl
(myenv) ➜  myenv pip install setuptools==75.8.1
...
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 75.8.0
    Uninstalling setuptools-75.8.0:
      Successfully uninstalled setuptools-75.8.0
Successfully installed setuptools-75.8.1
(myenv) ➜  myenv pip wheel --no-build-isolation pypika                                                
...
Building wheels for collected packages: pypika
  Building wheel for pypika (PEP 517) ... done
  Created wheel for pypika: filename=pypika-0.48.9-py2.py3-none-any.whl size=53769 sha256=8f667e0f5b564a903d5116cf150cf80fd2dc938db10c4ec485d070c8fb2607c7
  Stored in directory: /home/user/.cache/pip/wheels/3b/59/32/380c4d773968f655e18971a44e5a99f96dc2b9fb76287dae59
Successfully built pypika

🌍 Your Environment

Operating System:

  
Linux
  

Output of bazel version:

  
7.3.1
  

Rules_python version:

  
0.39.0
  

Anything else relevant?

[aignas]

#2410 should include this, however no one is working on this yet.

I am inclined to close as duplicate, but will leave open if you want to submit fixes to the repo rules.

EDIT: Some extra thoughts on this:

• pip is vendoring all of the deps with itself, so what gets loaded should be deterministic?
• If it is not, I wonder if the PYTHON path that gets constructed in the pypi_repo_utils is non-deterministic.
• Maybe we should remove pip from hermetic toolchain, which get rid of those vendored extra deps. Maybe having a completely clean interpreter help with determinism here, but it is somewhat a spray and pray strategy...
• are there any blocking issues that would hold you from testing if the bug is reproducible with the latest rules_python version?

[linzhp]

pip is vendoring all of the deps with itself, so what gets loaded should be deterministic?

From the documentation, it's isolated but I don't think it's deterministic. Pip reads build-system.requires key in the pyproject.toml to see if it requires specific version of setuptools. In the example, it specifies "setuptools ~= 58.0". So pip will get the latest version the range of 58.0.x. In the case of PyPika, it doesn't specify setuptools version, so pip will get the latest version available in PyPI. The "latest" mean different version if we build at different times.

are there any blocking issues that would hold you from testing if the bug is reproducible with the latest rules_python version?

I just verify it in rules_python 1.2.0. I can see that pip still creates pypika-0.48.9-py2.py3-none-any.whl, which indicates that it's using setuptools 75.8.1 or later, while the setuptools that comes with rules_python is 70.0.0: https://github.com/bazelbuild/rules_python/blob/a816962e509311c23230730b4b28f9d52a229949/python/private/pypi/deps.bzl#L79

[chrisirhc]

Looked into this a bit, and I'm guessing the behavior can sort of be explained by https://discuss.python.org/t/pip-build-isolation-without-installing-build-dependencies/52145 . I was able to reproduce it in https://github.com/chrisirhc/rules_python-repro/tree/chua/pypika-setuptools , and you can see the behavior in the logs by using the command in readme.

To "fix" this behavior and make pypika installation make use of the pypi deps, you'd need to add a --no-build-isolation to extra_pip_args. This stops pip from creating a new empty virtual environment for the sdist build, and lets the build use the setuptools that's available to it.

It's not ideal to remove build isolation. I guess what you'd want/expect is:

• have the build isolation of a virtual environment
• the virtual environment to sort of "inherit" the available dependencies (e.g. setuptools from rules_python/python/private/pypi/deps.bzl)

I don't think that mode exists right now, at least based on the above discussion link.

Another workaround to get stable reproducible builds would be to avoid sdist — build a wheel and upload it into an internal repo and point the requirements to a direct uri of that wheel.

[groodt]

This doesn't even seem like a bug? It seems like a bug fix by setuptools. Now that setuptools is following PEP 491 for wheel names, it's expected behaviour.

pip is an implementation detail of rules_python and it is a build frontend, not a build backend. sdist declare their own build backend configuration in their pyproject.toml

e.g

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

So Im struggling to understand the issue or impact here. If you want to avoid sdist head-aches, don't use sdist. Otherwise, this is behaving as expected and intended by the package authors tbh.

I'd like to understand:

• The impact of this issue. It seems very minor to me. If you have any sdist in your dependencies, all bets are off anyway
• Why is this a bug with rules_python and not with PyPika

I think this is a very minor bug, so I will chat with the other maintainers, but probably suggest we close it in a week as "won't fix".

[linzhp]

Why is this a bug with rules_python and not with PyPika

Without any changes to the repo, rules_python can pick up any version of setuptools and build a wheel with a different sha256. This violates the hermeticity, which should be provided by Bazel/rules_python.

The impact of this issue.

In a large repo that depends on the sha256 of all targets to determine what to build and test on CI, and which service to redeploy BazelCon 2019, this led to indeterminism.

[groodt]

There are no guarantees of determinism for any sdist. It goes beyond setuptools. Any sdist build can (and often is) non-deterministic. This is true inside or outside bazel. This is true even if no dependencies change.

The support for wheels can be considered deterministic because the download and unpack steps can be made to be deterministic.

But sdist builds can't be guaranteed to be deterministic because they involve arbitrary code execution. And the builds are also not managed by bazel.

So if you require full determinism for third-party python dependencies, you can achieve this by sourcing the third-party dependencies as wheel-only in the dependency closure.

So we should close this as duplicate of #2410 because it's just another of the known issues with sdist. It's not a regression.