Skip to content

Conversation

tobiasdiez
Copy link
Contributor

@tobiasdiez tobiasdiez commented Sep 25, 2025

Minimal version of #40327 to fix the issue mentioned in https://groups.google.com/g/sage-release/c/7LhB72fmw6k/m/QINoQj8YAgAJ, as well as strengthen the CI-inclusion pattern to hopefully catch similar issues earlier next time.

📝 Checklist

  • The title is concise and informative.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation and checked the documentation preview.

⌛ Dependencies

Copy link

github-actions bot commented Sep 25, 2025

Documentation preview for this PR (built with commit b806949; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@dimpase
Copy link
Member

dimpase commented Sep 25, 2025

these tests should be adjusted/removed:

**********************************************************************
File "src/sage/misc/package.py", line 277, in sage.misc.package.pkg_sources
Failed example:
    sage_conf_info = L['sage_conf']
Exception raised:
    Traceback (most recent call last):
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 733, in _run
        self.compile_and_execute(example, compiler, test.globs)
        ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 1157, in compile_and_execute
        exec(compiled, globs)
        ~~~~^^^^^^^^^^^^^^^^^
      File "<doctest sage.misc.package.pkg_sources[3]>", line 1, in <module>
        sage_conf_info = L['sage_conf']
                         ~^^^^^^^^^^^^^
    KeyError: 'sage_conf'
**********************************************************************
File "src/sage/misc/package.py", line 278, in sage.misc.package.pkg_sources
Failed example:
    sage_conf_info.type
Exception raised:
    Traceback (most recent call last):
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 733, in _run
        self.compile_and_execute(example, compiler, test.globs)
        ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 1157, in compile_and_execute
        exec(compiled, globs)
        ~~~~^^^^^^^^^^^^^^^^^
      File "<doctest sage.misc.package.pkg_sources[4]>", line 1, in <module>
        sage_conf_info.type
        ^^^^^^^^^^^^^^
    NameError: name 'sage_conf_info' is not defined
**********************************************************************
File "src/sage/misc/package.py", line 280, in sage.misc.package.pkg_sources
Failed example:
    sage_conf_info.is_installed()
Exception raised:
    Traceback (most recent call last):
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 733, in _run
        self.compile_and_execute(example, compiler, test.globs)
        ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 1157, in compile_and_execute   
        exec(compiled, globs) 
        ~~~~^^^^^^^^^^^^^^^^^
      File "<doctest sage.misc.package.pkg_sources[5]>", line 1, in <module>
        sage_conf_info.is_installed()
        ^^^^^^^^^^^^^^
    NameError: name 'sage_conf_info' is not defined
**********************************************************************
File "src/sage/misc/package.py", line 282, in sage.misc.package.pkg_sources
Failed example:
    sage_conf_info.source
Exception raised:
    Traceback (most recent call last):
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 733, in _run
        self.compile_and_execute(example, compiler, test.globs)
        ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Volumes/dima/software/sage/src/sage/doctest/forker.py", line 1157, in compile_and_execute
        exec(compiled, globs)
        ~~~~^^^^^^^^^^^^^^^^^
      File "<doctest sage.misc.package.pkg_sources[6]>", line 1, in <module>
        sage_conf_info.source
        ^^^^^^^^^^^^^^
    NameError: name 'sage_conf_info' is not defined

@tobiasdiez
Copy link
Contributor Author

Thanks, fixed now.

Copy link
Member

@dimpase dimpase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vbraun
Copy link
Member

vbraun commented Sep 27, 2025

Building from a clean checkout still fails, the one thing that this was supposed to fix

src/bin/sage -t --long --warn-long 30.0 --random-seed=123 src/sage/env.py
**********************************************************************
File "src/sage/env.py", line 22, in sage.env
Failed example:
    out = check_output([sys.executable, "-c", cmd], env=env).decode().strip()   # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/release/Sage/src/sage/doctest/forker.py", line 733, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/release/Sage/src/sage/doctest/forker.py", line 1157, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.env[8]>", line 1, in <module>
        out = check_output([sys.executable, "-c", cmd], env=env).decode().strip()   # long time
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/release/Sage/local/var/lib/sage/venv-python3.12.5/lib/python3.12/subprocess.py", line 466, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/release/Sage/local/var/lib/sage/venv-python3.12.5/lib/python3.12/subprocess.py", line 571, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/home/release/Sage/local/var/lib/sage/venv-python3.12.5/bin/python3', '-c', "from sage.all import SAGE_ROOT, SAGE_LOCAL;from os.path import samefile;s1 = samefile(SAGE_ROOT, '/home/release/Sage');s2 = samefile(SAGE_LOCAL, '/home/release/Sage/local');print(s1 and s2);"]' returned non-zero exit status 1.
**********************************************************************
File "src/sage/env.py", line 23, in sage.env
Failed example:
    out == "True"                                                               # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/release/Sage/src/sage/doctest/forker.py", line 733, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/release/Sage/src/sage/doctest/forker.py", line 1157, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.env[9]>", line 1, in <module>
        out == "True"                                                               # long time
        ^^^
    NameError: name 'out' is not defined
**********************************************************************
1 item had failures:
   2 of  11 in sage.env
    [47 tests, 2 failures, 2.69s wall]
----------------------------------------------------------------------
src/bin/sage -t --long --warn-long 30.0 --random-seed=123 src/sage/env.py  # 2 doctests failed
----------------------------------------------------------------------

giphy

@dimpase
Copy link
Member

dimpase commented Sep 27, 2025

the build obviously works, no?

a long doctest fails.

@dimpase
Copy link
Member

dimpase commented Sep 27, 2025

@vbraun - can we fix the long tests in a follow up?

@vbraun
Copy link
Member

vbraun commented Sep 27, 2025

Just fix it on this PR

@tobiasdiez
Copy link
Contributor Author

tobiasdiez commented Sep 28, 2025

I fail to see how this is related to the sage_conf spkg, but this test is anyway a source of current issues (eg #40869) and doesn't pass on a pure meson build either. Moreover, the sage.all import in pure python is anyway tested with pytest on CI. And that importing sage.all works in pure python, is tested with pytest. So I propose we just remove the test.

@user202729
Copy link
Contributor

user202729 commented Sep 28, 2025

to make debugging easier, I suggest changing the test to, say

sage: # long time
sage: out = subprocess.run([sys.executable, "-c", cmd], env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, text=True, encoding='u8', errors='replace')
sage: out.returncode
0
sage: out.stdout.strip()
"True"
sage: out.stderr.strip()
""

of course the reason why this makes things easier to debug is that all stdout are swallowed when there's an exception, so we avoid the exception and manually check the returncode, even if the process fails we can still see what the stdout/stderr were.

If you or @dimpase can't reproduce it, you'll still need to wait for @vbraun to run it though.

@tobiasdiez tobiasdiez force-pushed the remove_sage_conf_minimial branch from 7698a33 to b806949 Compare September 28, 2025 03:21
@vbraun
Copy link
Member

vbraun commented Sep 28, 2025

Did anybody here actually try building Sage from a clean repo checkout? Its not that hard and I would be surprised if you wouldn't hit the same test failure.

And a big 👎 from me for just deleting the test

Verify that importing ``sage.all`` works in Sage's Python without any
``SAGE_`` environment variables, and has the same ``SAGE_ROOT`` and
``SAGE_LOCAL`` (see also :issue:`29446`).

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

In effect we are slaving away for a macOS buildbot environment, an environment nobody develops on.
No two environments are the same.

A typical macOS machine is such a mess that often a much easier than building Sage tasks fail. I have to tell my students to create a separate account to do such work. (this week one of them could not manage to build a C program linking to GMP because of, well, environment!)

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

I'm not able to build sage-the-distro for over two years now, so yes - I'm very sorry, but I cannot test this directly myself.

Use something like docker or podman on a fast machine to create, say, a Gentoo image with all the Sage packages installed.
(Gentoo doesn't carry a few Python packages used by Sage, but installing them is very fast)

Then testing Sage-the-distro only takes about as long as testing sagelib and docbuild.

I also imagine we should run our CI with such an image, I don't see why our CI always builds Sage distro from configurations missing lots of packages.

OK that would explain why your PRs keep breaking it; Whats the actual issue? It really is not that hard, fire up a container with the prereqs and build?

I can re-run the PR after the unittest is updatde to emit better diagnostics

@vbraun
Copy link
Member

vbraun commented Sep 28, 2025

Did anybody here actually try building Sage from a clean repo checkout? Its not that hard and I would be surprised if you wouldn't hit the same test failure.

@vbraun on Linux the previous interaction of this PR, before beta5, builts and passes ALL the relevant tests, INCLUDING the sage.env ones (which gave you the reason for not merging it)

Incremental build, or build from a clean repo checkout?

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

@vbraun after

git clean -fdx && ./bootstrap 

as clean as it gets

@vbraun
Copy link
Member

vbraun commented Sep 28, 2025

I'm getting

sage: env = {k:v for (k,v) in os.environ.items() if not k.startswith("SAGE_")}
sage: from subprocess import check_output
sage: check_output([sys.executable, "-c", "from sage.all import SAGE_ROOT, SAGE_LOCAL; print([SAGE_ROOT, SAGE_LOCAL])"], env=env)
b"[None, '/usr']\n"

@vbraun
Copy link
Member

vbraun commented Sep 28, 2025

Also, do you have a global sage_conf install somewhere? src/sage/env.py contains:

    if value is None:
        try:
            import sage_conf
            value = getattr(sage_conf, key, None)
        except ImportError:
            try:
                import sage.config
                value = getattr(sage.config, key, None)
            except ImportError:
                pass

Also, if we want to get rid of sage_conf then that import would need to be removed

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

I'm getting

sage: env = {k:v for (k,v) in os.environ.items() if not k.startswith("SAGE_")}
sage: from subprocess import check_output
sage: check_output([sys.executable, "-c", "from sage.all import SAGE_ROOT, SAGE_LOCAL; print([SAGE_ROOT, SAGE_LOCAL])"], env=env)
b"[None, '/usr']\n"

How do you manage to get SAGE_LOCAL=='/usr' ? Is Sage's python3 for you /usr/bin/python3? (which is 3.9 on macOS). Do you force is? I think it should have been rejected, as the minimum we (nomarally) support is 3.11. Or is it Linux? (I don't get problems with this test on Linux, do you?)

On macOS I get b"[None, '/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.13']\n"

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

Also, do you have a global sage_conf install somewhere? src/sage/env.py contains:

    if value is None:
        try:
            import sage_conf
            value = getattr(sage_conf, key, None)
        except ImportError:
            try:
                import sage.config
                value = getattr(sage.config, key, None)
            except ImportError:
                pass

Also, if we want to get rid of sage_conf then that import would need to be removed

Good catch! Indeed, a copy of old sage_conf allowed this test to pass for me on Linux. Some much for the environment....

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

@tobiasdiez one has to remove stuff in pkgs/sage-conf*, not (only) in pkgs/sage_conf*.

Also, you seem to have forgotten to revert the Volker's revert, duh...

e32c83f47730235cefc70fac86d733ec

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

@tobiasdiez - I've posted a rebase and a fix on tobiasdiez#17 - please verify it's OK
It seems to work for me

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

cause it's a rebase, I'm hesitant to push it directly

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

with my PR, docs don't build:

[sagemath_doc_html-none] [spkg-install] [developer] updating environment: [new config] 22 added, 0 changed, 0 removed
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/coding_basics.rst:9: WARNING: undefined label: 'spkg' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/coding_basics.rst:1045: WARNING: undefined label: 'spkg_furo' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/coding_in_other.rst:7: WARNING: undefined label: 'spkg' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/downstream.rst:28: WARNING: undefined label: 'spkg' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/git_setup.rst:16: WARNING: undefined label: 'spkg_git' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/github.rst:39: WARNING: undefined label: 'spkg_github_cli' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/packaging.rst:99: WARNING: undefined label: 'spkg_pip' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/packaging.rst:887: WARNING: undefined label: 'spkg' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:102: WARNING: undefined label: 'spkg_flint' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:119: WARNING: undefined label: 'spkg_cvxopt' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:122: WARNING: undefined label: 'spkg_cvxpy' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:125: WARNING: undefined label: 'spkg_cypari' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:128: WARNING: undefined label: 'spkg_cysignals' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:131: WARNING: undefined label: 'spkg_flint' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:134: WARNING: undefined label: 'spkg_gmpy2' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:137: WARNING: undefined label: 'spkg_ipywidgets' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:140: WARNING: undefined label: 'spkg_matplotlib' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:143: WARNING: undefined label: 'spkg_mpmath' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:146: WARNING: undefined label: 'spkg_networkx' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:149: WARNING: undefined label: 'spkg_numpy' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:152: WARNING: undefined label: 'spkg_pplpy' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:155: WARNING: undefined label: 'spkg_rpy2' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:158: WARNING: undefined label: 'spkg_scipy' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:161: WARNING: undefined label: 'spkg_sympy' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:209: WARNING: undefined label: 'spkg_ecl' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:217: WARNING: undefined label: 'spkg_gap' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:225: WARNING: undefined label: 'spkg_giac' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:233: WARNING: undefined label: 'spkg_maxima' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:237: WARNING: undefined label: 'spkg_meson' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:241: WARNING: undefined label: 'spkg_pari' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:245: WARNING: undefined label: 'spkg_polymake' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:249: WARNING: undefined label: 'spkg_ppl' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:253: WARNING: undefined label: 'spkg_qepcad' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:257: WARNING: undefined label: 'spkg_scip' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:261: WARNING: undefined label: 'spkg_singular' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] /home/dima/software/sage/build/sage-distro/src/doc/en/developer/sage_manuals.rst:265: WARNING: undefined label: 'spkg_soplex' [ref.ref]
[sagemath_doc_html-none] [spkg-install] [developer] The HTML pages are in src/doc/html/en/developer.
[sagemath_doc_html-none] [spkg-install] Error building the documentation.
[sagemath_doc_html-none] [spkg-install] Traceback (most recent call last):
[sagemath_doc_html-none] [spkg-install]   File "/home/dima/software/sage/src/build-docs.py", line 11, in <module>
[sagemath_doc_html-none] [spkg-install]     main()
[sagemath_doc_html-none] [spkg-install]     ~~~~^^
[sagemath_doc_html-none] [spkg-install]   File "/home/dima/software/sage/src/sage_docbuild/__main__.py", line 548, in main
[sagemath_doc_html-none] [spkg-install]     build()
[sagemath_doc_html-none] [spkg-install]     ~~~~~^^
[sagemath_doc_html-none] [spkg-install]   File "/home/dima/software/sage/src/sage_docbuild/builders.py", line 142, in f
[sagemath_doc_html-none] [spkg-install]     runsphinx()
[sagemath_doc_html-none] [spkg-install]     ~~~~~~~~~^^
[sagemath_doc_html-none] [spkg-install]   File "/home/dima/software/sage/src/sage_docbuild/sphinxbuild.py", line 324, in runsphinx
[sagemath_doc_html-none] [spkg-install]     sys.stderr.raise_errors()
[sagemath_doc_html-none] [spkg-install]     ~~~~~~~~~~~~~~~~~~~~~~~^^
[sagemath_doc_html-none] [spkg-install]   File "/home/dima/software/sage/src/sage_docbuild/sphinxbuild.py", line 255, in raise_errors
[sagemath_doc_html-none] [spkg-install]     raise OSError(self._error)
[sagemath_doc_html-none] [spkg-install] OSError: /home/dima/software/sage/build/sage-distro/src/doc/en/developer/coding_basics.rst:9: WARNING: undefined label: 'spkg' [ref.ref]

there is also crud related to sage_conf, some of it looks serious:
there is one in .github/workflows/dist.yml, one in src/bin/sage-venv-config, in configure.ac, etc

@dimpase
Copy link
Member

dimpase commented Sep 28, 2025

at least tests appear to work, including the one in src/sage/env.py (which I restored in my PR)

@tobiasdiez
Copy link
Contributor Author

I'm getting

sage: env = {k:v for (k,v) in os.environ.items() if not k.startswith("SAGE_")}
sage: from subprocess import check_output
sage: check_output([sys.executable, "-c", "from sage.all import SAGE_ROOT, SAGE_LOCAL; print([SAGE_ROOT, SAGE_LOCAL])"], env=env)
b"[None, '/usr']\n"

That SAGE_ROOT is None is the expected output. Meson doesn't embed SAGE_ROOT in config.py (in contrast to sage_conf) and this is by design since sagelib should not depend on SAGE_ROOT and it is now also relocatable. So there is no point in providing SAGE_ROOT.

The problem is that sage -t (or more precisely src/bin/sage-env-config.in) is setting the SAGE_ROOT env variable. Thus, in the doctest env SAGE_ROOT is set to some directory that might or might not be correct, but it is none if you just use from sage.all import SAGE_ROOT from plain python. The same thing happens in the pure meson build, where the test fails with the same error. The reason that it worked for sage-the-distro before this PR is that it installed a second copy of config.py that actually provided a SAGE_ROOT variable.

Based on this, I see the following options:

  1. Remove the test, as it tests outdated behavior that is no longer true
  2. Modify the test so that it always checks that SAGE_ROOT is None when using from sage.all import SAGE_ROOT from plain python
  3. Stop setting SAGE_ROOT in the sage-env files

Do you have a preference, or another suggestion on how to fix this?

The problem with SAGE_LOCAL is similar.

@tobiasdiez
Copy link
Contributor Author

tobiasdiez commented Sep 29, 2025

Also, do you have a global sage_conf install somewhere? src/sage/env.py contains:

    if value is None:
        try:
            import sage_conf
            value = getattr(sage_conf, key, None)
        except ImportError:
            try:
                import sage.config
                value = getattr(sage.config, key, None)
            except ImportError:
                pass

Also, if we want to get rid of sage_conf then that import would need to be removed

A complete removal of sage_conf is done in #40327. Shall we close this PR here and continue talking there?

@tobiasdiez tobiasdiez mentioned this pull request Sep 29, 2025
5 tasks
@dimpase
Copy link
Member

dimpase commented Sep 29, 2025

I don't mind to move over to #40327

@tobiasdiez
Copy link
Contributor Author

@orlitzky Do you have any suggestion on how to solve the issue #40882 (comment) (since you mentioned your struggles with this test in #40869 (comment))

@orlitzky
Copy link
Contributor

At the risk of becoming the "delete the test" guy, I think we should... delete the test.

The purpose of the test is to ensure that from sage.all import * from python will import the SAGE_ROOT and SAGE_LOCAL variables from sagelib, and that those variables will have the same values as they do in a sage shell (always from the sage distro, back when this was written).

But SAGE_ROOT and SAGE_LOCAL are meaningless within sagelib. There's no reason to define them, and certainly no reason for them to agree with whatever values the sage distro has chosen. Why test it?

@tobiasdiez
Copy link
Contributor Author

Thanks for your assessment. I've now deleted the test in #40327. Let's continue there.

@vbraun
Copy link
Member

vbraun commented Sep 30, 2025

You are misrepresenting the test. First and foremost, it tests that you can import sage.all without any environment variables set. This actually failed here and saved us on this PR from shipping yet another broken release. Its not just a nice-to-have. Big -1 to just deleting it.

@orlitzky
Copy link
Contributor

You are misrepresenting the test. First and foremost, it tests that you can import sage.all without any environment variables set. This actually failed here and saved us on this PR from shipping yet another broken release. Its not just a nice-to-have. Big -1 to just deleting it.

I must be missing something. What's broken, other than the test? If we really can't import sage.all without SAGE_* set, it would be a lot simpler to keep only that useful portion of the test with something like

import os

old_env = {}
for k in os.environ:
    if k.startswith("SAGE_"):
        # Save SAGE_FOO variables
        old_env[k] = os.environ[k]
        os.environ[k] = ""

try:
    import sage.all
finally:
    # dict merge; restore SAGE_FOO
    os.environ = os.environ | old_env

It looks a lot nicer in pytest without all the shenanigans, and without the part that compares meaningless variables.

@dimpase
Copy link
Member

dimpase commented Sep 30, 2025

@orlitzky @vbraun - it seems that we see an insufficient separation between sage the distro and sagelib as far as the sage.env test is concerned. SAGE_ROOT is not needed for sagelib itself. Or is it?
See e.g. src/sage/doctest/control.py - does it work without SAGE_ROOT set?

@tobiasdiez
Copy link
Contributor Author

tobiasdiez commented Sep 30, 2025

You are misrepresenting the test. First and foremost, it tests that you can import sage.all without any environment variables set. This actually failed here and saved us on this PR from shipping yet another broken release. Its not just a nice-to-have. Big -1 to just deleting it.

What error did you encounter with this PR when importing sage.all?

I would be very surprised if this got broken, especially since CI runs pytest (outside of a sage env) and pytest loads sage.all at

sage/conftest.py

Lines 367 to 368 in f4adc25

# Inject sage.all into each doctest
import sage.repl.ipython_kernel.all_jupyter
without any SAGE_ env variables.

You also reported above that the sage.all import succeeded: #40882 (comment)

@orlitzky
Copy link
Contributor

@orlitzky @vbraun - it seems that we see an insufficient separation between sage the distro and sagelib as far as the sage.env test is concerned. SAGE_ROOT is not needed for sagelib itself. Or is it? See e.g. src/sage/doctest/control.py - does it work without SAGE_ROOT set?

It tries to import SAGE_ROOT from sage.env, so in that sense it needs the variable to exist to avoid an import error, but the way it's used is completely optional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants