Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Handle broken metadata by discarding package, or give a useful error #2119

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

khink
Copy link

@khink khink commented Jul 31, 2024

Fixes #2118, but no tests yet (hence WIP).

After this change, a requirements file is generated by discarding the problematic package.

Now, with this in requirements.txt:

django-debug-toolbar
django-geojson
wagtail>=4.2,<4.3

I get:

$ env/bin/pip-compile --resolver=backtracking --generate-hashes --allow-unsafe --output-file requirements.txt requirements.in
    error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [18 lines of output]
        /home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/dist.py:268: UserWarning: Unknown distribution option: 'test_suite'
          warnings.warn(msg)
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/tmp/pip-resolve-o2r8fhgd/geojson_90f4efbe8a2247f9a1883c03e4988381/setup.py", line 6, in <module>
            setup(name          = "geojson",
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
            return distutils.core.setup(**attrs)
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 146, in setup
            _setup_distribution = dist = klass(attrs)
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 289, in __init__
            self.metadata.version = self._normalize_version(self.metadata.version)
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 325, in _normalize_version
            normalized = str(Version(version))
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
            match = self._regex.search(version)
        TypeError: cannot use a string pattern on a bytes-like object
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
Discarding django-geojson (from -r requirements.in (line 11)) to proceed the resolution.
It looks like it has a release with broken metadata, causing this resolver to fail.
You may try to pin that package's version (probably to a recent one), so the resolver doesn't consider the broken release.

The exception is still shown. Also we log a warning to tell the user the package was discarded.

The way to find the offending pacakge is crude (regex in error message string).

When the regex search for package name fails, we re-raise the error so no requirements file is generated.

In that case we log an error, which is hopefully helpful.

Contributor checklist
  • Included tests for the changes.
  • PR title is short, clear, and ready to be included in the user-facing changelog.
Maintainer checklist
  • Verified one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
  • Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

Fixes jazzband#2118.

Now, with this in requirements.txt:

    django-debug-toolbar
    django-geojson
    wagtail>=4.2,<4.3

I get:

    $ env/bin/pip-compile --resolver=backtracking --generate-hashes --allow-unsafe --output-file requirements.txt requirements.in
        error: subprocess-exited-with-error

        × python setup.py egg_info did not run successfully.
        │ exit code: 1
        ╰─> [18 lines of output]
            /home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/dist.py:268: UserWarning: Unknown distribution option: 'test_suite'
              warnings.warn(msg)
            Traceback (most recent call last):
              File "<string>", line 2, in <module>
              File "<pip-setuptools-caller>", line 34, in <module>
              File "/tmp/pip-resolve-o2r8fhgd/geojson_90f4efbe8a2247f9a1883c03e4988381/setup.py", line 6, in <module>
                setup(name          = "geojson",
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
                return distutils.core.setup(**attrs)
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 146, in setup
                _setup_distribution = dist = klass(attrs)
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 289, in __init__
                self.metadata.version = self._normalize_version(self.metadata.version)
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 325, in _normalize_version
                normalized = str(Version(version))
              File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
                match = self._regex.search(version)
            TypeError: cannot use a string pattern on a bytes-like object
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
    Discarding django-geojson (from -r requirements.in (line 11)) to proceed the resolution.
    It looks like it has a release with broken metadata, causing this resolver to fail.
    You may try to pin that package's version (probably to a recent one), so the resolver doesn't consider the broken release.

The exception is still shown. Also we log a warning to tell the user the package was discarded.

The way to find the offending pacakge is crude (regex in error message string).

When the regex search for package name fails, we re-raise the error so no requirements file is generated.

In that case we log an error, which is hopefully helpful.
@khink khink changed the title WIP: Try handling broken metadata in resolver, or give a useful error WIP: Handle broken metadata by discarding package, or give a useful error Jul 31, 2024
@chrysle
Copy link
Contributor

chrysle commented Dec 27, 2024

Could you try to rebase, please?

@khink
Copy link
Author

khink commented Dec 27, 2024

Why? Is someone planning on actually working on it and this is the blocker?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"TypeError: cannot use a string pattern on a bytes-like object" while installing django-geojson
2 participants