Anaconda environment issues for the Desktop app

[jarjk]

windows

$ winget install miniconda3 # note: also tried with basic anaconda

$ conda env create -f .\environment.yml
Do you accept the Terms of Service (ToS) for https://repo.anaconda.com/pkgs/main? [(a)ccept/(r)eject/(v)iew]: a
Do you accept the Terms of Service (ToS) for https://repo.anaconda.com/pkgs/r? [(a)ccept/(r)eject/(v)iew]: a
Do you accept the Terms of Service (ToS) for https://repo.anaconda.com/pkgs/msys2? [(a)ccept/(r)eject/(v)iew]: a
3 channel Terms of Service accepted
Retrieving notices: done
Channels:
 - conda-forge
 - pytorch
 - defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - gunicorn=22.0.0

Current channels:

  - https://conda.anaconda.org/conda-forge
  - https://conda.anaconda.org/pytorch
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - https://repo.anaconda.com/pkgs/msys2

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

---

(arch) linux

$ paru -S anaconda (from chaotic-aur)
...

$ conda env create -f environment.yml                                                                 (master) 17:49
<frozen importlib._bootstrap>:488: Warning: OpenSSL 3's legacy provider failed to load. Legacy algorithms will not be available. If you need those algorithms, check your OpenSSL configuration.
Do you accept the Terms of Service (ToS) for https://repo.anaconda.com/pkgs/main? [(a)ccept/(r)eject/(v)iew]: a
Do you accept the Terms of Service (ToS) for https://repo.anaconda.com/pkgs/r? [(a)ccept/(r)eject/(v)iew]: a
2 channel Terms of Service accepted
Retrieving notices: done
Channels:
 - conda-forge
 - pytorch
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
    current version: 25.11.1
    latest version: 26.1.1

Please update conda by running

    $ conda update -n base -c defaults conda



Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/matahary/.conda/envs/WasteDetection/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt', '--exists-action=b']
Pip subprocess output:
Collecting flask-cors==4.0.1
  Downloading Flask_Cors-4.0.1-py2.py3-none-any.whl (14 kB)
Collecting geoalchemy2==0.14.6
  Downloading GeoAlchemy2-0.14.6-py3-none-any.whl (73 kB)
Collecting jsonmerge==1.9.0
  Downloading jsonmerge-1.9.0.tar.gz (32 kB)
Collecting opencv-python-headless==4.5.3.56
  Downloading opencv_python_headless-4.5.3.56-cp38-cp38-manylinux2014_x86_64.whl (37.1 MB)
Collecting psycopg2-binary==2.9.9
  Downloading psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting sqlalchemy==1.4.53
  Downloading SQLAlchemy-1.4.53-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting ttkbootstrap==1.7.3
  Downloading ttkbootstrap-1.7.3-py3-none-any.whl (131 kB)
Requirement already satisfied: Flask>=0.9 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (2.2.5)
Requirement already satisfied: packaging in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from geoalchemy2==0.14.6->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 2)) (26.0)
Collecting jsonschema>2.4.0
  Downloading jsonschema-4.23.0-py3-none-any.whl (88 kB)
Requirement already satisfied: numpy>=1.17.3 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from opencv-python-headless==4.5.3.56->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 4)) (1.24.4)
Requirement already satisfied: greenlet!=0.4.17 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from sqlalchemy==1.4.53->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 6)) (3.0.3)
Requirement already satisfied: pillow>=8.2.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from ttkbootstrap==1.7.3->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 7)) (9.2.0)
Requirement already satisfied: itsdangerous>=2.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (2.2.0)
Requirement already satisfied: importlib-metadata>=3.6.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (8.5.0)
Requirement already satisfied: Jinja2>=3.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (3.1.4)
Requirement already satisfied: click>=8.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (8.1.7)
Requirement already satisfied: Werkzeug>=2.2.2 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (3.0.6)
Requirement already satisfied: zipp>=3.20 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from importlib-metadata>=3.6.0->Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (3.21.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from Jinja2>=3.0->Flask>=0.9->flask-cors==4.0.1->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 1)) (2.1.5)
Collecting referencing>=0.28.4
  Downloading referencing-0.35.1-py3-none-any.whl (26 kB)
Requirement already satisfied: importlib-resources>=1.4.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from jsonschema>2.4.0->jsonmerge==1.9.0->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 3)) (6.4.5)
Collecting rpds-py>=0.7.1
  Downloading rpds_py-0.20.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (360 kB)
Collecting jsonschema-specifications>=2023.03.6
  Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl (18 kB)
Collecting pkgutil-resolve-name>=1.3.10
  Downloading pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Requirement already satisfied: attrs>=22.2.0 in /home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages (from jsonschema>2.4.0->jsonmerge==1.9.0->-r /home/matahary/code/wastedetection/condaenv.aj45z3us.requirements.txt (line 3)) (24.2.0)
Building wheels for collected packages: jsonmerge
  Building wheel for jsonmerge (setup.py): started
  Building wheel for jsonmerge (setup.py): finished with status 'done'
  Created wheel for jsonmerge: filename=jsonmerge-1.9.0-py3-none-any.whl size=18628 sha256=e556946bac063ef9126cf00d2e8c66164b30d872e358ceaf23dd77d17010ce03
  Stored in directory: /home/matahary/.cache/pip/wheels/82/c3/9a/421a7eabc04d18b19ee10e43646bea935e518eb88a6e5e9df7
Successfully built jsonmerge
Installing collected packages: rpds-py, referencing, pkgutil-resolve-name, jsonschema-specifications, sqlalchemy, jsonschema, ttkbootstrap, psycopg2-binary, opencv-python-headless, jsonmerge, geoalchemy2, flask-cors
  Attempting uninstall: sqlalchemy
    Found existing installation: SQLAlchemy 2.0.32
    Uninstalling SQLAlchemy-2.0.32:
      Successfully uninstalled SQLAlchemy-2.0.32
Successfully installed flask-cors-4.0.1 geoalchemy2-0.14.6 jsonmerge-1.9.0 jsonschema-4.23.0 jsonschema-specifications-2023.12.1 opencv-python-headless-4.5.3.56 pkgutil-resolve-name-1.3.10 psycopg2-binary-2.9.9 referencing-0.35.1 rpds-py-0.20.1 sqlalchemy-1.4.53 ttkbootstrap-1.7.3

done
#
# To activate this environment, use
#
#     $ conda activate WasteDetection
#
# To deactivate an active environment, use
#
#     $ conda deactivate

~/c/wastedetection►conda activate WasteDetection                                                              834.149s (master) 18:03
<frozen importlib._bootstrap>:488: Warning: OpenSSL 3's legacy provider failed to load. Legacy algorithms will not be available. If you need those algorithms, check your OpenSSL configuration.
~/c/wastedetection[1]►ls                                                                       (WasteDetection) 3.571s (master) 18:07
desktop_app/  environment.yml  model/      README.md           run_server_app.py  unit_tests/
Dockerfile    LICENSE          notebooks/  run_desktop_app.py  server_app/        web_app/
~/c/wastedetection►python run_desktop_app.py                                                       (WasteDetection) (master) 18:07
Traceback (most recent call last):
  File "run_desktop_app.py", line 4, in <module>
    import desktop_app.src.controller as controller
  File "/home/matahary/code/wastedetection/desktop_app/src/controller.py", line 13, in <module>
    from model import model
  File "/home/matahary/code/wastedetection/model/model.py", line 23, in <module>
    from model.unet import UNET
  File "/home/matahary/code/wastedetection/model/unet.py", line 1, in <module>
    import torch
  File "/home/matahary/.conda/envs/WasteDetection/lib/python3.8/site-packages/torch/__init__.py", line 290, in <module>
    from torch._C import *  # noqa: F403
ImportError: libtorch_cpu.so: cannot enable executable stack as shared object requires: Invalid argument

[mcserep]

Hi @jarjk,
Thanks for sharing your experience.

Windows

gunicorn is the production grade web server utilized by the web backend. It is for Linux only. You may safely remove that from the environment.yml file, as it is not required for anything else. Even the web backend can be executed with e.g. Flask's own webserver, although it is not designed for production environments. (But with minimal load in a local environment, it is fine.)

We will update the Anaconda environment and exclude this package, and install it only in the Docker image, which is intended for production usage.

Linux

On Linux, ELF binaries and shared libraries can be marked as either requiring or not requiring an executable stack via a special PT_GNU_STACK program header. When a library is marked as needing an executable stack, the dynamic linker tries to make the process stack executable when loading it. On hardened systems or newer combinations of kernels and C libraries, that attempt can fail with “cannot enable executable stack as shared object requires: Invalid argument,” which is the import error you see with libtorch_cpu.so.

Recent distros and toolchains are tightening executable‑stack handling, so libraries built (or with dependencies built) using GCC features that require trampolines can end up marked as needing an executable stack. On systems where the kernel/loader refuses to honor that request, importing PyTorch or libtorch fails with the ImportError.

This is an issue for modern Linux systems and only surfaced the past week in our CI system during Docker image build. A temporary workaround for this is to remove the execstack flag from libtorch, as it does not really require that. We do that in our CI: 762717b

The proper solution is to upgrade the pytorch version in the project, for which we already have an internal ticket since last week.

[mcserep]

Hi @jarjk,

In df5630c, we have decomposed 3 separate Conda environments for the 3 Python subprojects (Desktop Application, Server Application, Web Backend Application), so now gunicorn is only required for the Web Backend Application.

This resolves your first problem in this issue.

[jarjk]

excellent, I'll give it a try, although it does work when manually removing gunicorn

[jarjk]

yup, windows desktop env works as expected!