ci: publish the weekly sweep's smoke timings under a globbable name - #182
Merged
Merged
Conversation
`RunReport.write()` emits a consolidated `smoke_timings.json` into every report dir (PyAutoHands d2a22f4), and PyAutoHeart#167 published it from the PR gate as `smoke-timings-<python-version>`. The weekly sweep runs through a different body — workspace-validation.yml — which got no such upload, so the widest timing sample the organism produces survived only inside the per-leg `results-*` zips, under no name a consumer could glob for, and expired with them at 30 days. Confirmed on weekly run 32688412626: 109 artifacts, not one `smoke-timings-*`. Mirror the gate's named upload into the `run_scripts` and `run_notebooks` legs. The name carries the leg — `smoke-timings-<leg>-<project>-<directory>` — because the gate has two legs per run and this body has ~50; that is the same (project, directory) pair the sibling `results-*` names already rely on to stay unique. Only the JSON is uploaded (the report dir already ships as `results-*`) and deliberately with no `retention-days`, so the dataset keeps the repo's full default window and outlives the zips it came from. The new artifacts stay out of the `results-*` namespace on purpose: that is what `analyze` downloads and hands to `aggregate_results.py`, which globs `**/*.json` and skips this sidecar by name. smoke-tests.yml is untouched — no change to PR-gate behaviour. Six tests in test_workflow_wiring.py pin the names, the two fail-open guards, the notebook leg's no-notebooks gate, the absent retention-days, and the namespace separation. Full suite: 641 passed. Closes #181
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #181.
Summary
RunReport.write()has emitted a consolidatedsmoke_timings.jsoninto everyreport dir since PyAutoHands
d2a22f4, and #167 published it from the PR gate(
smoke-tests.yml) assmoke-timings-<python-version>. The weekly sweep runsthrough a different body —
workspace-validation.yml— which never got thatupload. So the widest timing sample the organism produces survived only inside
the per-leg
results-*zips, under no name a consumer could glob for, and expiredwith them at 30 days.
Verified against the 2026-08-24 weekly run
32688412626: 109 artifacts, not onesmoke-timings-*.Decision: option (a)
The prompt offered two routes. This takes (a) — mirror the PR gate's named
upload — rather than (b), widening the deferred phase-3 board ingester's
contract to also glob
results-*.(b) was rejected because there is nothing to widen: outside
smoke-tests.ymlandits tests, nothing in this repo references
smoke-timingsat all. The ingesterdoes not exist yet, so (b) would mean writing a promise instead of a fix — and
would leave the timings expiring with the 30-day
results-*retention. (a) alsoleaves that future ingester with one glob instead of two.
API Changes
None. No Python source is touched — this is CI wiring plus tests.
smoke-tests.ymlis untouched, so PR-gate behaviour is unchanged.
The one externally-visible change is additive: a new family of artifact names,
smoke-timings-{scripts,notebooks}-<project>-<directory>, published by the weeklysweep.
Design notes
results-*payload. The artifact name isthe contract a timing ingester globs on.
results-*is a different contract —it is what the
analyzejob downloads and hands toaggregate_results.py,which globs
**/*.jsonand skips this sidecar by name. Publishing timingsunder a
results-*name would aim them at the one consumer that deliberatelyexcludes them.
fixed name is safe there; this body runs ~98 legs in a single weekly run. The
(project, directory)pair is exactly what the siblingresults-*namesalready rely on the script matrix to keep unique.
results-*;duplicating ~98 of them weekly buys nothing.
retention-days, deliberately (theresults-*siblings use 30). Thedataset is the point, matching feat: upload the smoke report dir so per-script timings persist #167's reasoning — the timings now keep the
repo's full default window and outlive the zips they were extracted from.
if: always()+if-no-files-found: ignoreare both load-bearing: afailing leg still has timings worth keeping, and a leg that dies before the
first report write must not turn "we collected no timings" into "the weekly
sweep is red".
Testing
Heart suite on the branch: 641 passed. Six new tests in
tests/test_workflow_wiring.pypin the artifact names, both fail-open guards, thenotebook leg's no-notebooks gate, the absent
retention-days, and theresults-*/smoke-timings-*namespace separation.End-to-end: a real weekly sweep on this branch
Dispatched
workspace-smoke.ymlon this branch →run 32902243623.
referenced_workflowsresolves toworkspace-validation.yml@5829bbb, so itexercised this branch's body, not
main's.One-to-one across every executed leg, both
scriptsandnotebooks.No artifact-name collisions — not one upload step failed in the whole run,
which is the proof, since
upload-artifact@v4hard-fails with a 409 on aduplicate name.
Retention split confirmed live, e.g. the
autolens-imagingscripts leg:results-scripts-autolens-imagingsmoke-timings-scripts-autolens-imagingThat sweep's four red legs, and why none is this change
Note this dispatch shares the head sha with the PR, so its checks are attached
here. It is the scheduled weekly channel, not a PR gate.
run_scripts (3.12, autolens_test, multi_dataset)jax_likelihood/shared_preloads.py→TIMEOUT (300s)block_until_readycompile stall in that familyrun_notebooks (3.12, howtofit, chapter_1_introduction)Failed to download action 'actions/download-artifact@v4'— DNS,internal-api.service.iad.github.netrun_notebooks (3.12, howtogalaxy, chapter_2_modeling)run_notebooks (3.12, howtogalaxy, chapter_4_scaling_up_galaxies)smoke / analyzeOn the scripts failure: 16 of 17 scripts passed, and the bimodality is visible
in the same log — siblings
mge.py(11.7s),rectangular_mge.py(20.7s) andmge_group.py(49.3s) all passed comfortably. The failure is at step 9(
run_python.py); the new step 12 then ran anyway and succeeded —smoke-timings-scripts-autolens_test-multi_dataset, 690 bytes, artifact9583834034. That is
if: always()behaving as intended: a failing leg stillyields its timings,
TIMEOUTrow and cap included.On the three notebook failures: all three died inside "Prepare all required
actions" — before checkout and before any step, mine included — within one
~3-minute window (21:53–21:57).
actions/download-artifactis pre-existing inthat job (it fetches the generated notebooks); this diff adds only an
upload-artifactstep, andupload-artifactdownloaded fine in the same jobs'siblings. Those three legs therefore produced neither a
results-*nor asmoke-timings-*, which is why the 98/98 pairing above is unaffected.