ci: publish the weekly sweep's smoke timings under a globbable name

[Jammy2211]

Overview

RunReport.write() has emitted a consolidated smoke_timings.json in every
report dir since PyAutoHands d2a22f4, and PyAutoHeart#167 published it from
the PR gate (smoke-tests.yml) as smoke-timings-<python-version>. The
weekly sweep goes through a different body — workspace-validation.yml —
which got no such upload, so the widest timing sample the organism produces
persists only inside the per-leg results-* zips, under no name a consumer
can glob for, and expires with those zips at 30 days.

Verified on the 2026-08-24 weekly run 32688412626 (success, 136 jobs): all
109 artifacts are results-* / notebooks-* / workspace-validation-report
— not one smoke-timings-*. The legs do emit the file (the
results-scripts-autolens-imaging artifact grew 2 files / 3,950 B → 3 files /
5,306 B between the 2026-08-19 and 2026-08-24 runs); only the naming is
missing.

This takes option (a) from the prompt — mirror the PR gate's named upload
into workspace-validation.yml — rather than (b), deferring the whole contract
to a phase-3 ingester that does not exist yet. (a) makes the weekly data
discoverable now and leaves the future ingester with one glob
(smoke-timings-*) instead of two.

Plan

• Add a named smoke-timings-* upload to workspace-validation.yml's
  run_scripts and run_notebooks legs, alongside (not replacing) the
  existing results-* uploads.
• Carry the leg in the artifact name — smoke-timings-<leg>-<project>-<directory>
  — because the weekly run has ~50 legs where the PR gate has two, so the PR
  gate's fixed name would collide here. That is the same (project, directory)
  pair the sibling results-* names already rely on to stay unique.
• Upload only smoke_timings.json, not the whole report dir: the report dir
  already ships as results-*, and duplicating ~50 of them weekly buys nothing.
• Give the timings no retention-days, so they keep the repo's full default
  window (the results-* zips they came from expire at 30 days) — the same
  reasoning the PR gate's upload records.
• Keep the new artifacts out of the results-* namespace, which the analyze
  job downloads and feeds to aggregate_results.py — that consumer skips the
  timing sidecar by name.
• Pin all of the above in tests/test_workflow_wiring.py, next to the existing
  PR-gate timing tests.
• Leave smoke-tests.yml untouched: no change to PR-gate behaviour.

Detailed implementation plan

Work Classification

Library (Heart's own CI wiring) — no workspace repo is touched.

Affected Repositories

• PyAutoHeart (primary)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoHeart	main	clean

Other branches on the remote: main, claude/health-agent-implementation-x3z8w6,
archive/condemned/pyautoheart-feature-autonerves-verify-install. No conflict —
nothing else claims PyAutoHeart.

Suggested branch: claude/weekly-smoke-timings-naming-012fuj

Worktree root: ~/Code/PyAutoLabs-wt/weekly-smoke-timings-naming/ (local-dev
only; this run is a web-github session working from the session clone).

Implementation Steps

1. .github/workflows/workspace-validation.yml, job run_scripts — after the
   existing Upload script results step, add:

   - name: Upload the per-script timings (scripts leg) under a globbable name
     if: always()
     uses: actions/upload-artifact@v4
     with:
       name: smoke-timings-scripts-${{ matrix.project.name }}-${{ matrix.project.directory }}
       path: workspace/**/smoke_timings.json
       if-no-files-found: ignore

   with a comment block recording why it is a separate artifact (the name is
   the ingester's contract, and results-* is the aggregate consumer's), why
   the name carries the leg (~50 legs vs the gate's two), why only the JSON,
   and why there is no retention-days.

2. Same job run_notebooks, after Upload notebook results — the mirror step,
   named smoke-timings-notebooks-…, keeping that leg's existing gate
   always() && steps.gate.outputs.run == 'true' (the *_test workspaces
   publish no notebooks, so those legs never execute).

3. tests/test_workflow_wiring.py — six tests appended beside the existing
   PR-gate timing tests:

   • test_validation_body_publishes_the_timing_dataset — both legs upload
     smoke_timings.json under a smoke-timings-* name.
   • test_validation_timing_artifact_names_carry_the_leg — the exact
     collision-free name, per leg.
   • test_validation_timing_upload_cannot_fail_the_weekly_sweep — always()
     • if-no-files-found: ignore.
   • test_validation_notebook_timing_upload_keeps_the_no_notebooks_gate.
   • test_validation_timing_dataset_keeps_full_default_retention — no
     retention-days key.
   • test_timing_artifacts_stay_out_of_the_aggregate_namespace — analyze
     still downloads pattern: results-*, and neither new name starts with
     results-.

Key Files

• .github/workflows/workspace-validation.yml — the weekly sweep's body; the
  two new upload steps.
• .github/workflows/smoke-tests.yml — the PR gate. Untouched; its
  smoke-timings-<python-version> upload is the pattern being mirrored.
• tests/test_workflow_wiring.py — the wiring pins.
• PyAutoHands autohands/result_collector.py — RunReport.write(), the
  producer of smoke_timings.json. Not modified.

Trade-offs

• (a) vs (b). (b) would defer the whole thing to the phase-3 Heart-board
  ingester by widening its contract to glob results-* too. That contract
  does not exist yet (nothing in PyAutoHeart references smoke-timings outside
  smoke-tests.yml and its tests), so (b) means writing a promise instead of a
  fix, and leaves the timings expiring at the results-* 30-day retention. (a)
  is also symmetric with the PR gate, which is the shape a future ingester
  wants.
• Separate artifact vs widening the results-* payload. Keeping them apart
  costs one extra upload step per leg but keeps the two consumer contracts
  clean — aggregate_results.py globs **/*.json under results-* and
  excludes the sidecar by name; feeding it a second copy is the wrong
  direction.
• mode=release also gets the upload. run_scripts runs in both modes and
  the step is unconditional, so a release-integration run publishes its timings
  too. Harmless and arguably useful; the project set is a subset there, so no
  new collision risk.

Testing

• python3 -m pytest tests/ — full Heart suite, on the branch.
• The only true end-to-end proof is a weekly workspace-smoke.yml run (or a
  manual workflow_dispatch of it): the artifact list should gain
  smoke-timings-scripts-* / smoke-timings-notebooks-* entries with no
  duplicate-name failures across the ~50 legs.

Original Prompt

Click to expand starting prompt

The weekly smoke run's timings land in results-* under no discoverable name

Type: maintenance
Target: pyautoheart
Repos:

• @PyAutoHeart
  Difficulty: small
  Autonomy: safe
  Priority: low
  Status: draft
  Filed: 2026-08-24

Follow-on gap from complete/2026/08/smoke-timings-dataset.md (PyAutoHands
d2a22f4 + PyAutoHeart#167 3df42b5, merged 2026-08-24). All facts below
verified 2026-08-24 via the Actions API.

The gap

RunReport.write() now emits smoke_timings.json (schema smoke_timings/1)
in every report dir, but the named upload
(smoke-timings-<python-version>, if: always(), if-no-files-found: ignore)
was added only to the reusable PR-gate workflow
@PyAutoHeart/.github/workflows/smoke-tests.yml.

The weekly workspace-smoke.yml run goes through
@PyAutoHeart/.github/workflows/workspace-validation.yml, which has no such
named upload — e.g. run 32688412626 (2026-08-24 04:00 UTC, success,
136 jobs).

Its run_scripts / generate_notebooks legs do emit the file now. Verified
differentially: the results-scripts-autolens-imaging artifact grew from
2 files / 3,950 bytes (2026-08-19, run 32277952488) to 3 files /
5,306 bytes (2026-08-24, PyAutoHands at d2a22f4). So the weekly timing
data — by far the widest sample the organism produces — persists only inside
the per-leg results-* zips, under no name a consumer can glob for.

Task

Pick one and record the decision in the PR body:

• (a) Add the same named upload to workspace-validation.yml's script and
  notebook legs (if: always(), if-no-files-found: ignore). Simpler and
  symmetric with the PR gate — preferred unless (b) proves cheaper.
• (b) Explicitly commit the deferred phase-3 Heart-board timing
  ingester (the follow-up named in complete/2026/08/smoke-timings-dataset.md)
  to globbing smoke_timings.json out of results-* artifacts as well as
  smoke-timings-* ones, and say so in that contract.

Trap for (a): artifact-name collisions. The PR gate has one leg per python
version, so a single fixed name is safe there. The weekly run has ~50+ legs in
one run, so a fixed smoke-timings-<python-version> would collide — the name
must carry the leg's package/directory (mirroring how results-* is already
disambiguated, e.g. results-scripts-autolens-imaging).

Acceptance

• Weekly-run timing data is either published under a predictable
  smoke-timings-* name, or the ingester contract explicitly covers
  results-* — not left implicit either way.
• No change to PR-gate behaviour (smoke-tests.yml untouched, or provably
  equivalent).
• workspace-validation.yml stays green; no artifact-name collisions in a
  full weekly run.

[Jammy2211]

End-to-end verification: full weekly sweep on the branch

Dispatched workspace-smoke.yml on claude/weekly-smoke-timings-naming-012fuj →
run 32902243623.
referenced_workflows resolves to workspace-validation.yml@5829bbb, so the run
exercised this branch's body, not main's.

Artifact census — exact

203 total = 1   workspace-validation-report
          + 6   notebooks-<project>   (generate_notebooks — converts .py→.ipynb, runs nothing, so no timings by design)
          + 196 → 98 results-* / 98 smoke-timings-*

One-to-one pairing across every executed leg, both scripts and notebooks.
Baseline for comparison: the 2026-08-24 weekly run 32688412626 produced 109
artifacts and zero smoke-timings-*.

No artifact-name collisions. Not one upload step failed anywhere in the run —
which is the proof, since upload-artifact@v4 hard-fails with a 409 on a
duplicate name. The (project, directory) suffix holds across all ~98 legs.

Retention split confirmed live, e.g. the autolens-imaging scripts leg:

Artifact	Size	Expires
results-scripts-autolens-imaging	5,336 B	2026-09-24 (30d)
smoke-timings-scripts-autolens-imaging	1,452 B	2026-11-23 (default)

The timings outlive the report dirs they were extracted from by two months.

The one red leg is not this change

run_scripts (3.12, autolens_test, multi_dataset) failed — 16 of 17 scripts
passed, with:

scripts/multi_dataset/jax_likelihood/shared_preloads.py ...   TIMEOUT (300s)
##[error]Process completed with exit code 1.

That is the known intermittent XLA/block_until_ready compile stall in the
multi_dataset/jax_likelihood family, already filed. The bimodality signature is
visible in this same log — siblings mge.py (11.7s), rectangular_mge.py
(20.7s) and mge_group.py (49.3s) all passed comfortably.

It is not attributable to this diff, which touches only upload steps and tests:
the failure is at step 9 (run_python.py), and the new step 12 then ran anyway
and succeeded — smoke-timings-scripts-autolens_test-multi_dataset, 690
bytes, artifact 9583834034. That is if: always() behaving exactly as intended:
a failing leg still yields its timings, including the TIMEOUT row carrying the
cap it hit — precisely the data the future board ingester wants most.

Acceptance

• Weekly timing data published under a predictable smoke-timings-* name
• No change to PR-gate behaviour — smoke-tests.yml untouched
• No artifact-name collisions in a full weekly run
• Heart suite green on the branch (641 passed)

---

Generated by Claude Code

[Jammy2211]

Shipped

Merged as #182 → 9e3f1c8 (verified: branch is an ancestor of origin/main, 0 unmerged commits).

Decision recorded: option (a). The named smoke-timings-* upload was added to
workspace-validation.yml's run_scripts and run_notebooks legs, with the leg
carried in the artifact name. Option (b) was rejected because the phase-3 board
ingester it would have committed does not exist — nothing in this repo referenced
smoke-timings outside smoke-tests.yml and its tests — so (b) meant writing a
promise instead of a fix, and would have left the timings expiring with the
30-day results-* retention.

Acceptance

• Weekly-run timing data published under a predictable smoke-timings-* name.
  Proven on a real dispatched sweep (run 32902243623):
  203 artifacts = 1 report + 6 notebooks-<project> + 98 results-* / 98
  smoke-timings-*, one-to-one across every executed leg.
• No change to PR-gate behaviour. smoke-tests.yml untouched.
• No artifact-name collisions in a full weekly run. Not one upload step
  failed anywhere in the sweep — the proof, since upload-artifact@v4
  hard-fails with a 409 on a duplicate name.
• Heart suite green: 641 passed locally; PR gate green on both legs
  (pytest 3.12, pytest 3.13).

Bonus confirmed live: the timings now outlive the report dirs they came from —
smoke-timings-scripts-autolens-imaging expires 2026-11-23 against its
results-* sibling's 2026-09-24.

One finding worth carrying forward

That sweep's autolens_test/multi_dataset leg failed on
jax_likelihood/shared_preloads.py → TIMEOUT (300s), while siblings mge.py
(11.7s), rectangular_mge.py (20.7s) and mge_group.py (49.3s) passed
comfortably. That is the intermittent XLA/block_until_ready compile stall
tracked in draft/bug/ci/jax_vmap_jit_compile_stall.md — but
shared_preloads.py is a family member not previously on that ledger's list
(which named mge.py and rectangular_mge.py). Worth adding as a new data point.

Three further red legs in that sweep were GitHub infrastructure, not code: three
run_notebooks jobs died inside "Prepare all required actions" on a DNS failure
fetching actions/download-artifact@v4, all within one ~3-minute window.

---

Generated by Claude Code