OS X: some doctests fail with extra text "ninja: warning: Ignoring jobserver: Pipe-based protocol is not supported!"

[jhpalmieri]

Steps To Reproduce

Running doctests in OS X 15.6.1, Sage 10.8.beta3, with various homebrew packages installed, I see doctest failures of this sort:

**********************************************************************
File "src/sage/env.py", line 22, in sage.env
Failed example:
    out = check_output([sys.executable, "-c", cmd], env=env).decode().strip()   # long time
Expected nothing
Got:
    ninja: warning: Ignoring jobserver: Pipe-based protocol is not supported! [ --jobserver-fds=3,4 -j -- TEST_FLAGS=--long TEST_TARGET=testlong]
**********************************************************************

and

**********************************************************************
File "src/sage/misc/sagedoc.py", line 32, in sage.misc.sagedoc
Failed example:
    os.system("sage -c \"if 'sphinx' in sys.modules: sys.exit(1)\"")
Expected:
    0
Got:
    ninja: warning: Ignoring jobserver: Pipe-based protocol is not supported! [ --jobserver-fds=3,4 -j -- TEST_FLAGS=--long TEST_TARGET=testlong]
    0
**********************************************************************

and several other files.

Expected Behavior

Doctests should pass.

Actual Behavior

Doctests fail.

Additional Information

One solution would be to add the appropriate string to the do_fixup method in src/sage/doctest/parsing.py.

Environment

• OS: OS X 15.6.1, Xcode 26.0, with many homebrew packages
• Sage Version: 10.8.beta3

Checklist

• I have searched the existing issues for a bug report that matches the one I want to file, without success.
• I have read the documentation and troubleshoot guide

[tobiasdiez]

Probably a result of https://github.com/sagemath/sage/blob/83b52a79babd619fcdd4c9eeb5760500cdfb92db/src/sage/doctest/forker.py#L1886C18-L1886C36, at least I have never seen it in a non-make setup.

[orlitzky]

These run sage to run the doctest runner to run sage to run a shell to run sage to see what happens tests are lunacy to begin with, but maybe this can be swept under the rug by upgrading to ninja-1.13.1 which does support the jobserver protocol under GNU Make 4.4 or later.

[user202729]

These run sage to run the doctest runner to run sage to run a shell to run sage to see what happens tests are lunacy to begin with

I don't understand what's the issue. The best I can see is "if it raises an error, it can be hard to debug whether the bug is in the doctest runner or the underlying program", but the same thing can be said with any tester ever (e.g. if you use pytest, you get "run Python to run pytest to fork itself to run a shell to run sage", which means if you get a bug it might as well be a bug with pytest or an interference between pytest and sage in a shell), no?

[orlitzky]

Normally when you're running a test suite, you're testing the copy of the program you just built. Things like,

sage: os.system("sage -c \"if 'sphinx' in sys.modules: sys.exit(1)\"")

can jump out of the sandbox and test the wrong sage.

Sage running within the doctester has to guess how it's being run, and often mess with its own environment in order to test what it wants to test. One of the tests above:

    sage: env = {k:v for (k,v) in os.environ.items() if not k.startswith("SAGE_")}                                                                             
    sage: from subprocess import check_output                                                                                                                  
    sage: module_name = "sage.all"   # hide .all import from the linter                                                                                        
    sage: cmd  = f"from {module_name} import SAGE_ROOT, SAGE_LOCAL;"                                                                                           
    sage: cmd +=  "from os.path import samefile;"                                                                                                              
    sage: if SAGE_ROOT is None:                                                                                                                                
    ....:     cmd +=  "s1 = SAGE_ROOT is None;"                                                                                                                
    ....: else:                                                                                                                                                
    ....:     cmd += f"s1 = samefile(SAGE_ROOT, '{SAGE_ROOT}');"                                                                                               
    sage: cmd += f"s2 = samefile(SAGE_LOCAL, '{SAGE_LOCAL}');"                                                                                                 
    sage: cmd += "print(s1 and s2);"                                                                                                                           
    sage: out = check_output([sys.executable, "-c", cmd], env=env).decode().strip()   # long time                                                              
    sage: out == "True"                                                               # long time                                                              
    True   

Whatever it's doing would be a lot simpler if didn't have to overwrite the environment, try to ensure that we're running python and not sage, and pass an entire python file on the CLI as a string. (IIRC I wrote that and got it wrong at least once, while trying to fix the original that was also wrong.)

Particularly when it goes wrong (why is sage -c running ninja?) it's a lot easier to figure out why when you're not in an inception meme.

[jhpalmieri]

What do people think about the proposal in the ticket description:

One solution would be to add the appropriate string to the do_fixup method in src/sage/doctest/parsing.py.

[orlitzky]

What do people think about the proposal in the ticket description:

One solution would be to add the appropriate string to the do_fixup method in src/sage/doctest/parsing.py.

I'm not opposed to a quick fix, but I think we should understand how random output from the build system is winding up in your doctests first. I can't even imagine how ninja is getting involved, but if the ninja output can get slurped up, it's probably not the only one.

[tobiasdiez]

First question to answer this would be: when exactly does the error happens? When you run make test or also when you use sage -t? Is it still there if you don't use editable install?

[orlitzky]

xref ninja upgrade: #40922

[user202729]

ninja has always been running in editable install (if some file changes), no?

[orlitzky]

It runs whenever you rebuild something with meson, but sage -c shouldn't cause anything to be rebuilt... and certainly python -c (in the first test) shouldn't.

I guess make <something> could trigger it, but if we're going to blame that, we have to explain how output from the build system got interleaved with the doctest output two processes deep.

[jhpalmieri]

I see the error messages after doing ./bootstrap && ./configure && make testlong from a fresh Sage clone. This turns out to use homebrew's version of ninja, which is 1.13.1. I tried the upgrade at #40922 (together with ./configure --with-system-ninja_build=no) and I get the same error messages. When I run the tests with ./sage ..., I do not see the errors.

[tobiasdiez]

What's the output if you run it with MESONPY_EDITABLE_VERBOSE set? Also please try the non-editable install of sage (there should be a configure flag for that).