Installation issues on MacOS and poetry

[GeoffNN]

🐛 Bugs / Unexpected behaviors

I'm following the installation instructions on at https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md. The installation errors by saying torch isn't installed, although it is.
I'm using Python 3.10.9, and a virtualenv through poetry. I've specific constraints pytorch<2.2 and torchvision=0.16. When installing from github, I get the following error:

On the other hand running poetry run python -c "import torch; print(torch.__version__)" returns "2.1.0".

Instructions To Reproduce the Issue:

Please include the following (depending on what the issue is):

1. Any changes you made (git diff) or code you wrote
   N/A
2. The exact command(s) you ran:

poetry new test_dir_pytorch3d
cd test_dir_pytorch3d
poetry env use 3.10
poetry add torch=2.1.0 torchvision=0.16
poetry run python -c "import torch; print(torch.__version__)"
>>> 2.1.0
MACOSX_DEPLOYMENT_TARGET=10.14 CC=clang CXX=clang++ poetry add  "git+https://github.com/facebookresearch/pytorch3d.git"
>>> ModuleNotFoundError: No module named 'torch'

4. What you observed (including the full logs):

Unable to determine package info for path: /Users/.../.venv/src/pytorch3d

Command ['/var/folders/vm/.../.venv/bin/python', '-I', '-W', 'ignore', '-c', "import build\nimport build.env\nimport pyproject_hooks\n\nsource = '/Users/.../.venv/src/pytorch3d'\ndest = '/var/folders/vm/.../dist'\n\nwith build.env.DefaultIsolatedEnv() as env:\n    builder = build.ProjectBuilder(\n        source_dir=source,\n        python_executable=env.python_executable,\n        runner=pyproject_hooks.quiet_subprocess_runner,\n    )\n    env.install(builder.build_system_requires)\n    env.install(builder.get_requires_for_build('wheel'))\n    builder.metadata_path(dest)\n"] errored with the following return code 1

Error output:
Traceback (most recent call last):
  File "<string>", line 15, in <module>
  File "/private/var/folders/vm/.../.venv/lib/python3.12/site-packages/build/__init__.py", line 239, in get_requires_for_build
    with self._handle_backend(hook_name):
  File "/opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/private/var/folders/vm/.../.venv/lib/python3.12/site-packages/build/__init__.py", line 360, in _handle_backend
    raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

Fallback egg_info generation failed.

Command ['/var/folders/vm/.../.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1

Output:
Traceback (most recent call last):
  File "/Users/.../.venv/src/pytorch3d/setup.py", line 15, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

Note that I get the same error when I try MACOSX_DEPLOYMENT_TARGET=10.14 CC=clang CXX=clang++ poetry run pip install "git+https://github.com/facebookresearch/pytorch3d.git"

[GeoffNN]

Using the -vvv option on poetry add:

(...-py3.10) (base) ➜  ... git:(main) ✗ MACOSX_DEPLOYMENT_TARGET=10.14 CC=clang CXX=clang++ poetry run poetry add  "git+https://github.com/facebookresearch/pytorch3d.git@stable" -vvv
Loading configuration file /Users/.../Library/Application Support/pypoetry/config.toml
Using virtualenv: /Users/.../.../.venv
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /facebookresearch/pytorch3d.git/info/refs?service=git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/facebookresearch/pytorch3d.git at 'stable' to /Users/.../.../.venv/src/pytorch3d

  Stack trace:

  2  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:350 in _run
      348│                 output = ""
      349│             else:
    → 350│                 output = subprocess.check_output(
      351│                     cmd, stderr=stderr, env=env, text=True, **kwargs
      352│                 )

  1  /opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py:466 in check_output
       464│         kwargs['input'] = empty
       465│
    →  466│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       467│                **kwargs).stdout
       468│

  CalledProcessError

  Command '['/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/bin/python', '-I', '-W', 'ignore', '-c', "import build\nimport build.env\nimport pyproject_hooks\n\nsource = '/Users/.../.../.venv/src/pytorch3d'\ndest = '/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/dist'\n\nwith build.env.DefaultIsolatedEnv() as env:\n    builder = build.ProjectBuilder(\n        source_dir=source,\n        python_executable=env.python_executable,\n        runner=pyproject_hooks.quiet_subprocess_runner,\n    )\n    env.install(builder.build_system_requires)\n    env.install(builder.get_requires_for_build('wheel'))\n    builder.metadata_path(dest)\n"]' returned non-zero exit status 1.

  at /opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
       568│             raise
       569│         retcode = process.poll()
       570│         if check and retcode:
    →  571│             raise CalledProcessError(retcode, process.args,
       572│                                      output=stdout, stderr=stderr)
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
       574│
       575│

The following error occurred when trying to handle this error:


  Stack trace:

  4  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/info.py:589 in get_pep517_metadata
      587│                 *PEP517_META_BUILD_DEPS,
      588│             )
    → 589│             venv.run_python_script(pep517_meta_build_script)
      590│             info = PackageInfo.from_metadata(dest_dir)
      591│         except EnvCommandError as e:

  3  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:325 in run_python_script
      323│
      324│     def run_python_script(self, content: str, **kwargs: Any) -> str:
    → 325│         return self.run(
      326│             self._executable,
      327│             "-I",

  2  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:317 in run
      315│     def run(self, bin: str, *args: str, **kwargs: Any) -> str:
      316│         cmd = self.get_command_from_bin(bin) + list(args)
    → 317│         return self._run(cmd, **kwargs)
      318│
      319│     def run_pip(self, *args: str, **kwargs: Any) -> str:

  1  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/virtual_env.py:99 in _run
       97│     def _run(self, cmd: list[str], **kwargs: Any) -> str:
       98│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    →  99│         return super()._run(cmd, **kwargs)
      100│
      101│     def get_temp_environ(

  EnvCommandError

  Command ['/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/bin/python', '-I', '-W', 'ignore', '-c', "import build\nimport build.env\nimport pyproject_hooks\n\nsource = '/Users/.../.../.venv/src/pytorch3d'\ndest = '/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/dist'\n\nwith build.env.DefaultIsolatedEnv() as env:\n    builder = build.ProjectBuilder(\n        source_dir=source,\n        python_executable=env.python_executable,\n        runner=pyproject_hooks.quiet_subprocess_runner,\n    )\n    env.install(builder.build_system_requires)\n    env.install(builder.get_requires_for_build('wheel'))\n    builder.metadata_path(dest)\n"] errored with the following return code 1

  Error output:
  Traceback (most recent call last):
    File "<string>", line 15, in <module>
    File "/private/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/lib/python3.12/site-packages/build/__init__.py", line 239, in get_requires_for_build
      with self._handle_backend(hook_name):
    File "/opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 158, in __exit__
      self.gen.throw(value)
    File "/private/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/lib/python3.12/site-packages/build/__init__.py", line 360, in _handle_backend
      raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
  build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel


  at ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:354 in _run
      350│                 output = subprocess.check_output(
      351│                     cmd, stderr=stderr, env=env, text=True, **kwargs
      352│                 )
      353│         except CalledProcessError as e:
    → 354│             raise EnvCommandError(e)
      355│
      356│         return output
      357│
      358│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  2  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:350 in _run
      348│                 output = ""
      349│             else:
    → 350│                 output = subprocess.check_output(
      351│                     cmd, stderr=stderr, env=env, text=True, **kwargs
      352│                 )

  1  /opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py:466 in check_output
       464│         kwargs['input'] = empty
       465│
    →  466│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       467│                **kwargs).stdout
       468│

  CalledProcessError

  Command '['/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/bin/python', 'setup.py', 'egg_info']' returned non-zero exit status 1.

  at /opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
       568│             raise
       569│         retcode = process.poll()
       570│         if check and retcode:
    →  571│             raise CalledProcessError(retcode, process.args,
       572│                                      output=stdout, stderr=stderr)
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
       574│
       575│

The following error occurred when trying to handle this error:


  Stack trace:

  3  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/info.py:606 in get_pep517_metadata
      604│             os.chdir(path)
      605│             try:
    → 606│                 venv.run("python", "setup.py", "egg_info")
      607│                 info = PackageInfo.from_metadata(path)
      608│             except EnvCommandError as fbe:

  2  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:317 in run
      315│     def run(self, bin: str, *args: str, **kwargs: Any) -> str:
      316│         cmd = self.get_command_from_bin(bin) + list(args)
    → 317│         return self._run(cmd, **kwargs)
      318│
      319│     def run_pip(self, *args: str, **kwargs: Any) -> str:

  1  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/virtual_env.py:99 in _run
       97│     def _run(self, cmd: list[str], **kwargs: Any) -> str:
       98│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    →  99│         return super()._run(cmd, **kwargs)
      100│
      101│     def get_temp_environ(

  EnvCommandError

  Command ['/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1

  Output:
  Traceback (most recent call last):
    File "/Users/.../.../.venv/src/pytorch3d/setup.py", line 15, in <module>
      import torch
  ModuleNotFoundError: No module named 'torch'


  at ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:354 in _run
      350│                 output = subprocess.check_output(
      351│                     cmd, stderr=stderr, env=env, text=True, **kwargs
      352│                 )
      353│         except CalledProcessError as e:
    → 354│             raise EnvCommandError(e)
      355│
      356│         return output
      357│
      358│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  17  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:327 in run
       325│
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  16  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│

  15  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│

  14  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:473 in _run_command
       471│
       472│         if error is not None:
     → 473│             raise error
       474│
       475│         return terminate_event.exit_code

  13  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:457 in _run_command
       455│
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  12  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│
     → 117│         return self.execute(io) or 0
       118│
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

  11  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/command.py:61 in execute
        59│
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

  10  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/commands/add.py:160 in handle
       158│             return 0
       159│
     → 160│         requirements = self._determine_requirements(
       161│             packages,
       162│             allow_prereleases=self.option("allow-prereleases"),

   9  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/commands/init.py:375 in _determine_requirements
       373│
       374│         result = []
     → 375│         for requirement in self._parse_requirements(requires):
       376│             if "git" in requirement or "url" in requirement or "path" in requirement:
       377│                 result.append(requirement)

   8  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/commands/init.py:441 in _parse_requirements
       439│             cwd=cwd,
       440│         )
     → 441│         return [parser.parse(requirement) for requirement in requirements]
       442│
       443│     def _format_requirements(self, requirements: list[dict[str, str]]) -> Requirements:

   7  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/dependency_specification.py:89 in parse
        87│
        88│         specification = (
     →  89│             self._parse_url(requirement)
        90│             or self._parse_path(requirement)
        91│             or self._parse_simple(requirement)

   6  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/dependency_specification.py:149 in _parse_url
       147│
       148│         if url_parsed.scheme in GIT_URL_SCHEMES:
     → 149│             return self._parse_git_url(requirement)
       150│
       151│         if url_parsed.scheme in ["http", "https"]:

   5  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/dependency_specification.py:133 in _parse_git_url
       131│
       132│         source_root = self._env.path.joinpath("src") if self._env else None
     → 133│         package = self._direct_origin.get_package_from_vcs(
       134│             "git",
       135│             url=url.url,

   4  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/packages/direct_origin.py:106 in get_package_from_vcs
       104│             raise ValueError(f"Unsupported VCS dependency {vcs}")
       105│
     → 106│         return _get_package_from_git(
       107│             url=url,
       108│             branch=branch,

   3  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/packages/direct_origin.py:46 in _get_package_from_git
        44│         path = path.joinpath(subdirectory)
        45│
     →  46│     package = DirectOrigin.get_package_from_directory(path)
        47│     package._source_type = "git"
        48│     package._source_url = url

   2  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/packages/direct_origin.py:75 in get_package_from_directory
        73│     @classmethod
        74│     def get_package_from_directory(cls, directory: Path) -> Package:
     →  75│         return PackageInfo.from_directory(path=directory).to_package(root_dir=directory)
        76│
        77│     def get_package_from_url(self, url: str) -> Package:

   1  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/info.py:487 in from_directory
       485│                         info = cls.from_setup_files(path)
       486│                     else:
     → 487│                         info = get_pep517_metadata(path)
       488│                 except PackageInfoError:
       489│                     if not info:

  PackageInfoError

  Unable to determine package info for path: /Users/.../.../.venv/src/pytorch3d

  Command ['/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/bin/python', '-I', '-W', 'ignore', '-c', "import build\nimport build.env\nimport pyproject_hooks\n\nsource = '/Users/.../.../.venv/src/pytorch3d'\ndest = '/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/dist'\n\nwith build.env.DefaultIsolatedEnv() as env:\n    builder = build.ProjectBuilder(\n        source_dir=source,\n        python_executable=env.python_executable,\n        runner=pyproject_hooks.quiet_subprocess_runner,\n    )\n    env.install(builder.build_system_requires)\n    env.install(builder.get_requires_for_build('wheel'))\n    builder.metadata_path(dest)\n"] errored with the following return code 1

  Error output:
  Traceback (most recent call last):
    File "<string>", line 15, in <module>
    File "/private/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/lib/python3.12/site-packages/build/__init__.py", line 239, in get_requires_for_build
      with self._handle_backend(hook_name):
    File "/opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 158, in __exit__
      self.gen.throw(value)
    File "/private/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/lib/python3.12/site-packages/build/__init__.py", line 360, in _handle_backend
      raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
  build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

  Fallback egg_info generation failed.

  Command ['/var/folders/vm/xdqwyft13jlb8h5dsr8jfl300000gn/T/tmp6i2w_d2m/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1

  Output:
  Traceback (most recent call last):
    File "/Users/.../.../.venv/src/pytorch3d/setup.py", line 15, in <module>
      import torch
  ModuleNotFoundError: No module named 'torch'

  at ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/info.py:609 in get_pep517_metadata
      605│             try:
      606│                 venv.run("python", "setup.py", "egg_info")
      607│                 info = PackageInfo.from_metadata(path)
      608│             except EnvCommandError as fbe:
    → 609│                 raise PackageInfoError(
      610│                     path, e, "Fallback egg_info generation failed.", fbe
      611│                 )
      612│             finally:
      613│                 os.chdir(cwd)

[GeoffNN]

This is likely a duplicate of #1673 and #1419, apologies.