Park multi_dataset/jax_likelihood/mge.py out of the smoke gate - #262
Merged
Conversation
The script hangs to the 300s smoke cap. It hit 4/4 jobs on autolens_workspace_test#261 (Python 3.12 and 3.13, twice each including a re-run), while that PR's diff touches three files, none of which this gate executes, and the script itself has no reference to what that PR changes. Not cap margin: run standalone under its declared `ENV: jax full_datasets` profile it completes in 32s against a 300s budget, simulating its dataset from scratch as CI does. It stops making progress rather than running long. This is a new instance of a known, documented bug. Its sibling multi_dataset/jax_likelihood/delaunay.py is already quarantined in config/build/no_run.yaml as "18s when it passes; intermittent XLA compile stall, family-wide" (autolens_workspace_test#245) — the same signature, the same folder. The third sibling, shared_preloads.py, is already disabled in this file. mge.py was the last multi_dataset/jax_likelihood entry still in the PR gate. Deliberately NOT added to config/build/no_run.yaml. Every failure observed is at the 300s smoke cap; there is no evidence about behaviour at the 1800s release cap, and quarantining it there on a guess would remove real release coverage. The release side is left as-is. Re-enable when the #245 stall is fixed.
3 tasks
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
multi_dataset/jax_likelihood/mge.pyhangs to the 300s smoke cap, blocking unrelated PRs. This comments it out ofsmoke_tests.txtwith the reason recorded inline, matching how its two siblings were already handled.This is a new instance of a known, documented bug — not a new diagnosis.
config/build/no_run.yaml:87already quarantines its sibling:"18s when it passes; intermittent XLA compile stall, family-wide" is precisely the signature here — same folder, same shape.
mge.pywas the lastmulti_dataset/jax_likelihood/entry still in the PR gate;shared_preloads.pywas disabled in this file on 2026-07-22.Evidence
TIMEOUT (300s) multi_dataset/jax_likelihood/mge.py, always 23/24 passed.interferometer/nufft.py,interferometer/jax_likelihood/rectangular_sparse.py,scripts/CLAUDE.md); none is insmoke_tests.txt, so this gate never executes them.mge.pycontains no reference to transformers, NUFFT or pynufft.ENV: jax full_datasetsprofile it completes in 32s against a 300s budget, simulating its dataset from scratch exactly as CI does. It stops making progress rather than running long.mge.pyis the only script in the gate that touchesdataset/multi_dataset/lens_sersic, and the runner executes each script as its own subprocess, so no in-process state carries over.What this deliberately does not do
No
config/build/no_run.yamlentry. Every failure observed is at the 300s smoke cap. I have no evidence about behaviour at the 1800s release cap, and quarantining it there on a guess would drop real release coverage for a script that runs in 32s. The release side is untouched.No attempt to fix the stall. That is #245's job and is family-wide (it has also hit
ag_test rectangular.py), not something to solve inside a gate-list change.Test Plan
smoke_tests.txtchanges — one line commented out, no script or config editsmge.pywas the sole failureGenerated by Claude Code