Use default Python install, not jupyterlab install

[cm421]

Brought up in a customer ticket #118720 pointing out that the jupyter version of python isn't user-writable so the default python version for this image isn't usable out of the box.

$ which python
/opt/python/jupyter/bin/python
$ pip install scipy
...
Installing collected packages: numpy, scipy
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/opt/python/jupyter/lib/python3.12/site-packages/numpy'
Check the permissions.

This reverts a change to set whatever the PYTHON_VERSION is to be the default python version

[cm421]

I'm not sure why the tests are failing on quarto as my change shouldn't have bumped that. Though I see a lot of other PRs with failing tests so I'm going to assume that the failure isn't specific to my changes

[bschwedler]

I'm not sure why the tests are failing on quarto as my change shouldn't have bumped that. Though I see a lot of other PRs with failing tests so I'm going to assume that the failure isn't specific to my changes

I wonder if Quarto is looking for specific python or packages. 🤔

---

@ianpittwood Do you anticipate any issues changing the python version in the default path?

[ianpittwood]

I'm not sure why the tests are failing on quarto as my change shouldn't have bumped that. Though I see a lot of other PRs with failing tests so I'm going to assume that the failure isn't specific to my changes

I wonder if Quarto is looking for specific python or packages. 🤔

@ianpittwood Do you anticipate any issues changing the python version in the default path?

At some point, and I don't remember when, there was a request to have jupyter be on the PATH. Instead of removing that altogether, we could do ENV PATH="/opt/python/${PYTHON_VERSION}/bin:/opt/python/jupyter/bin:${PATH}" instead so the primary installation's python takes priority in the PATH and still leaves the jupyter binary exposed.

[cm421]

Thank you both! I've updated to keep both Jupyter and the default python version on PATH. Curious to see if maybe Quarto did want some python packages that were in the Jupyter install?

[bschwedler]

We will need to remove the --quiet flag from the quarto check command in the goss tests to determine why we are getting a non-zero exit code.
We should be able to iterate on this locally to track it down.

[bschwedler]

It looks like a missing yaml package:

± docker run --rm -it  ghcr.io/rstudio/r-session-complete:ubuntu2204-2025.05.1-513.pro3 /usr/local/bin/quarto check
Quarto 1.6.42
[✓] Checking environment information...
      Quarto cache location: /root/.cache/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.4.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.46.3: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.6.42
      Path: /usr/lib/rstudio-server/bin/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (external install)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/local/bin
      Version: 2025

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.6
      Path: /opt/python/3.12.6/bin/python3
      Jupyter: 5.8.1
      Kernels: python3, py3.11.10, py3.12.6

(|) Checking Jupyter engine render....Traceback (most recent call last):
  File "/usr/lib/rstudio-server/bin/quarto/share/jupyter/jupyter.py", line 21, in <module>
    from notebook import notebook_execute, RestartKernel
  File "/usr/lib/rstudio-server/bin/quarto/share/jupyter/notebook.py", line 15, in <module>
    from yaml import safe_load as parse_string
ModuleNotFoundError: No module named 'yaml'
[✓] Checking Jupyter engine render....OK

[bschwedler]

Ah! I think Quarto is why we were setting the jupyter python version as the default.

For the quarto check tests:

Initial error:

ModuleNotFoundError: No module named 'yaml'

After installing pyyaml it also required installing nbformat and nbclient

The quarto docs recommend installing jupyter.

---

Back to the original issue where the python version isn't writable. The site-packages directory won't be writable by normal users for any version of python, but that shouldn't prevent people from installing them in their home directories.

I attempted to install using the rstudio-server user, but there is no home directory, which I assume is because we are expecting /home to be mounted for the users.

ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/rstudio-server'
Check the permissions.

We should discuss the approach we want to take to fix the original issue.

[bschwedler]

We may want to set QUARTO_PYTHON globally for all users and still depend on the jupyter python installation: https://quarto.org/docs/projects/environment.html

[bschwedler]

@cm421 How do you think we should approach this given the requirements quarto has?

[cm421]

@bschwedler sorry for the delay, is using QUARTO_PYTHON like this what you were thinking? I'm not sure if it's supposed to be used with a venv like this but it dos build and it passes quarto check without issue and also allows users to install packages with the normal python install.

[bschwedler]

@cm421 I think this should work.

The use case that this would not cover is individual users calling quarto without the QUARTO_PYTHON being set in their environment, which would depend on their rc/profile files for their shell.
Do you think it's worth adding this to /etc/profile.d/ or /etc/bash.bashrc?

[cm421]

I think that QUARTO_PYTHON will be available to users using the container. At least it was when I tested adding a user checking it and then running quarto check to verify.

What surprises me with the user I created is that the PATH with the full python install was clobbered by the user's default shell/profile, so the python wasn't readily available. I thought this was working before, where setting PATH with quarto's python in the front was getting picked up in user sessions.

Does this seem like what you'd expect?

craig@db60cc51ad3b:/$ which quarto
/usr/local/bin/quarto
craig@db60cc51ad3b:/$ which python
craig@db60cc51ad3b:/$ echo $QUARTO_PYTHON
/opt/python/jupyter/bin
craig@db60cc51ad3b:/$ quarto check
Quarto 1.7.32
[✓] Checking environment information...
      Quarto cache location: /home/craig/.cache/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.6.3: OK
      Dart Sass version 1.85.1: OK
      Deno version 1.46.3: OK
      Typst version 0.13.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.7.32
      Path: /usr/lib/rstudio-server/bin/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (external install)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/local/bin
      Version: 2025

[✓] Checking Chrome Headless....................OK
      Chrome:  (not detected)

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.11
      Path: /opt/python/jupyter/bin/python3
      Jupyter: 5.9.1
      Kernels: python3, py3.12.11, py3.11.13

...

[bschwedler]

What surprises me with the user I created is that the PATH with the full python install was clobbered by the user's default shell/profile, so the python wasn't readily available. I thought this was working before, where setting PATH with quarto's python in the front was getting picked up in user sessions.

Does this seem like what you'd expect?

Perhaps we need to add this to /etc/paths as well?