port to python 3.14

[dimpase]

This is to collect info on the porting to Python 3.14, which is the system Python on Fedora 43,
at the Sage 10.8.beta4 moment

First of all, you need #40882
to unbreak the build,

--- a/build/pkgs/python3/spkg-configure.m4
+++ b/build/pkgs/python3/spkg-configure.m4
@@ -1,8 +1,8 @@
 SAGE_SPKG_CONFIGURE([python3], [
    m4_pushdef([MIN_VERSION],               [3.11.0])
    m4_pushdef([MIN_NONDEPRECATED_VERSION], [3.11.0])
-   m4_pushdef([LT_STABLE_VERSION],         [3.14.0])
-   m4_pushdef([LT_VERSION],                [3.14.0])
+   m4_pushdef([LT_STABLE_VERSION],         [3.15.0])
+   m4_pushdef([LT_VERSION],                [3.15.0])
    AC_ARG_WITH([python],
                [AS_HELP_STRING([--with-python=PYTHON3],
                                [Python 3 executable to use for the Sage venv; default: python3])])

to allow the use of Python 3.14 as the Python for the venv, and #40888 to allow building with Python 3.14

[dimpase]

here is an error while building sagelib

[sagelib-10.8.beta4] [spkg-install]   [564/2578] Compiling C object src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/meson-generated_src_sage_cpython_atexit.pyx.c.o
[sagelib-10.8.beta4] [spkg-install]   FAILED: [code=1] src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/meson-generated_src_sage_cpython_atexit.pyx.c.o
[sagelib-10.8.beta4] [spkg-install]   gcc -Isrc/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p -Isrc/sage/cpython -I../../src/sage/cpython -I/usr/include/python3.14 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c17 -O3 -g -O2 -fPIC -MD -MQ src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/meson-generated_src_sage_cpython_atexit.pyx.c.o -MF src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/meson-generated_src_sage_cpython_atexit.pyx.c.o.d -o src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/meson-generated_src_sage_cpython_atexit.pyx.c.o -c src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/src/sage/cpython/atexit.pyx.c
[sagelib-10.8.beta4] [spkg-install]   src/sage/cpython/atexit.cpython-314-x86_64-linux-gnu.so.p/src/sage/cpython/atexit.pyx.c:1130:29: error: unknown type name 'atexit_py_callback'; did you mean 'atexit_callback'?
...

The whole machinery of Python's atexit_callback has changed in 3.14. The 1st step is

--- a/src/sage/cpython/atexit.pyx
+++ b/src/sage/cpython/atexit.pyx
@@ -152,10 +152,13 @@ cdef extern from *:
     #undef _PyGC_FINALIZED
     #include "internal/pycore_interp.h"
     #include "internal/pycore_pystate.h"
+    // atexit_py_callback is gone on 3.14, which got Py_PACK_VERSION
+    #ifndef Py_PACK_VERSION
     #if PY_VERSION_HEX >= 0x030c0000
     // struct atexit_callback was renamed in 3.12 to atexit_py_callback
     #define atexit_callback atexit_py_callback
     #endif
+    #endif
     static atexit_callback ** _atexit_callbacks(PyObject *self) {
         PyInterpreterState *interp = _PyInterpreterState_GET();
         struct atexit_state state = interp->atexit;

The structure atexit_state has changed a lot in 3.14 - different names, different types...

@orlitzky @tornaria - are you familiar with this code at all?

[dimpase]

In all the pyproject.toml files (in pkgs/ and in SAGEROOT) one needs

--- a/pyproject.toml
+++ b/pyproject.toml
@@ -82,7 +82,7 @@ description = "Sage: Open Source Mathematics Software: Standard Python Library"
 dynamic = ["version"]
 license = { text = "GNU General Public License (GPL) v2 or later" }
 name = "sagemath"
-requires-python = ">=3.11, <3.14"
+requires-python = ">=3.11, <3.15"
 urls = { Homepage = "https://www.sagemath.org" }
 
 [project.optional-dependencies]

[dimpase]

@embray - hi, using internal APIs has finally bitten us in src/sage/cpython/atexit.pyx.

As you appear to have designed the latter, could you explain a bit what it does and why it's needed?

[user202729]

not sure if @embray still checks GitHub, so I'll try my best to guess.

what it does: it exposes a restore_atexit context manager such that if you do

with restore_atexit(run=True, clear=True): SOMETHING

and SOMETHING does atexit.register(whatever), then whatever is executed right when the context ends.

why it's needed: it's used at exactly one place in doctest/forker, with the explanation

            # multiprocessing.Process instances don't run exit
            # functions, so we run the functions added by doctests
            # when exiting this context.

since the problem is that multiprocessing.Process does not honor atexit, and according to python/cpython#114279, Python ≥ 3.13 multiprocessing.Process does honor atexit, I suspect something like the following in forker.py suffice (if tests pass) (with something appropriate to avoid the code duplication)

if sys.version_info() >= (3, 13):
	for it in range(N):
		doctests, extras = self._run(runner, options, results)
		runner.summarize(options.verbose)
		if runner.update_results(results):
			break
else:
	# multiprocessing.Process instances don't run exit
	# functions, so we run the functions added by doctests
	# when exiting this context.
	with restore_atexit(run=True):
		for it in range(N):
			doctests, extras = self._run(runner, options, results)
			runner.summarize(options.verbose)
			if runner.update_results(results):
				break

as for sage.cpython.atexit module, one can simply not provide that module in Python ≥ 3.14 i.e. importing it is an error. Users of Python ≤ 3.13 wouldn't have any backwards compatibility issue. (Although I don't expect any external user to have a need to import that. Hopefully.)

Related discussion:

• py3: restore_atexit context manager #24234
• atexit module: allow getting/setting list of handlers directly python/cpython#76263
• no way to introspect registered atexit handlers python/cpython#61388

it links to a StackOverflow answer https://stackoverflow.com/a/63029332/5267751 which might work, if just flat out remove the functionality doesn't.

[dimpase]

@user202729 - would you like to take a shot at fixing the atexit issue? (it should be doable on Python 3.13, so can be done separately)