Cyclic dependency between hatch-fancy-pypi-readme

[mgorny]

We can't package the new version of hatch-fancy-pypi-readme on Gentoo because it has a cyclic dependency between jsonschema and hatch-fancy-pypi-readme. Since jsonschema requires hatch-fancy-pypi-readme to build, it needs to be installed first. However, it doesn't work if it's installed prior to jsonschema and therefore it breaks the build of jsonschema:

Traceback (most recent call last):
  File "/usr/lib/python-exec/pypy3/gpep517", line 4, in <module>
    sys.exit(main())
  File "/usr/lib/pypy3.9/site-packages/gpep517/__main__.py", line 322, in main
    return func(args)
  File "/usr/lib/pypy3.9/site-packages/gpep517/__main__.py", line 122, in build_wheel
    print(build_wheel_impl(args, args.wheel_dir), file=out)
  File "/usr/lib/pypy3.9/site-packages/gpep517/__main__.py", line 105, in build_wheel_impl
    wheel_name = backend.build_wheel(str(wheel_dir), args.config_json)
  File "/usr/lib/pypy3.9/site-packages/hatchling/build.py", line 59, in build_wheel
    return os.path.basename(next(builder.build(wheel_directory, ['standard'])))
  File "/usr/lib/pypy3.9/site-packages/hatchling/builders/plugin/interface.py", line 80, in build
    self.metadata.validate_fields()
  File "/usr/lib/pypy3.9/site-packages/hatchling/metadata/core.py", line 191, in validate_fields
    _ = self.version
  File "/usr/lib/pypy3.9/site-packages/hatchling/metadata/core.py", line 85, in version
    self._set_version()
  File "/usr/lib/pypy3.9/site-packages/hatchling/metadata/core.py", line 173, in _set_version
    version = self.hatch.version.cached
  File "/usr/lib/pypy3.9/site-packages/hatchling/metadata/core.py", line 1256, in cached
    raise type(e)(f'Error getting the version from source `{self.source.PLUGIN_NAME}`: {e}') from None
ModuleNotFoundError: Error getting the version from source `vcs`: No module named 'jsonschema'

[hynek]

this has already been fixed 22.6 – see #13

I kinda hoped an immediate Sunday re-release saves me from hearing from all distros 😬

[mgorny]

Unfortunately, it isn't. #13 was about hatch-fancy-pypi-readme depending on itself.

[hynek]

Ugh that's a problem too? I'll see what I can do, but that will take a bit longer.

[mgorny]

You can also reproduce it using pip install --no-binary :all: (note it's super-slow):

$ pip install --no-binary ':all:' hatch-fancy-pypi-readme
Collecting hatch-fancy-pypi-readme
  Downloading hatch_fancy_pypi_readme-22.6.0.tar.gz (27 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: hatchling in ./.venv/lib/python3.11/site-packages (from hatch-fancy-pypi-readme) (1.9.0)
Collecting jsonschema
  Downloading jsonschema-4.16.0.tar.gz (292 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 292.4/292.4 kB 4.2 MB/s eta 0:00:00
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 2
  ╰─> [146 lines of output]
      Collecting hatchling
        Using cached hatchling-1.9.0.tar.gz (55 kB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: still running...
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting hatch-vcs
        Downloading hatch_vcs-0.2.0.tar.gz (9.3 kB)
        Installing build dependencies: started
        Installing build dependencies: still running...
        Installing build dependencies: still running...
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting hatch-fancy-pypi-readme
        Using cached hatch_fancy_pypi_readme-22.6.0.tar.gz (27 kB)
        Installing build dependencies: started
        Installing build dependencies: still running...
        Installing build dependencies: still running...
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting editables>=0.3
        Using cached editables-0.3.tar.gz (4.7 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting packaging>=21.3
        Using cached packaging-21.3.tar.gz (84 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting pathspec>=0.10.1
        Using cached pathspec-0.10.1.tar.gz (43 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting pluggy>=1.0.0
        Using cached pluggy-1.0.0.tar.gz (51 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting setuptools-scm>=6.4.0
        Using cached setuptools_scm-7.0.5.tar.gz (70 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting jsonschema
        Using cached jsonschema-4.16.0.tar.gz (292 kB)
      ERROR: Exception:
      Traceback (most recent call last):
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
          status = run_func(*args)
                   ^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
          return func(self, options, args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 369, in run
          requirement_set = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
          result = self._result = resolver.resolve(
                                  ^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
          state = resolution.resolve(requirements, max_rounds=max_rounds)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 373, in resolve
          failure_causes = self._attempt_to_pin_criterion(name)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
          criteria = self._get_updated_criteria(candidate)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 204, in _get_updated_criteria
          self._add_to_criteria(criteria, requirement, parent=candidate)
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
          if not criterion.candidates:
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
          return bool(self._sequence)
                 ^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
          return any(self)
                 ^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
          return (c for c in iterator if id(c) not in self._incompatible_ids)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
          candidate = func()
                      ^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
          self._link_candidate_cache[link] = LinkCandidate(
                                             ^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
          super().__init__(
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
          self.dist = self._prepare()
                      ^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
          dist = self._prepare_distribution()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
          return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 438, in prepare_linked_requirement
          return self._prepare_linked_requirement(req, parallel_builds)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 524, in _prepare_linked_requirement
          dist = _get_prepared_distribution(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 67, in _get_prepared_distribution
          with build_tracker.track(req):
        File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__
          return next(self.gen)
                 ^^^^^^^^^^^^^^
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/operations/build/build_tracker.py", line 122, in track
          self.add(req)
        File "/tmp/.venv/lib/python3.11/site-packages/pip/_internal/operations/build/build_tracker.py", line 92, in add
          raise LookupError(message)
      LookupError: https://files.pythonhosted.org/packages/cf/54/8923ba38b5145f2359d57e5516715392491d674c83f446cd4cd133eeb4d6/jsonschema-4.16.0.tar.gz (from https://pypi.org/simple/jsonschema/) (requires-python:>=3.7) is already being built: jsonschema from https://files.pythonhosted.org/packages/cf/54/8923ba38b5145f2359d57e5516715392491d674c83f446cd4cd133eeb4d6/jsonschema-4.16.0.tar.gz (from hatch-fancy-pypi-readme)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[hynek]

60e1d1f should fix this. It would good if you could confirm that if I push a 22.7 today, there won't be a 22.8 tomorrow.

[mgorny]

I can confirm that with the commit applied as a patch on top of 22.6.0, I can build hatch-fancy-pypi-readme and then jsonschema successfully. Thanks!

For the record, I think you could use jsonschema if you made it as an extra dependency and made the code handle it being missing gracefully (e.g. skip validation if that's its only use).

[hynek]

Cool, thanks.

Not gonna add extra complexity for this given the patch was even negative…I live to serve. 🤪

22.7 is on PyPI