Commit 6eaf6ed
test: smoke runs every script, not a hand-maintained allowlist
smoke_tests.txt was an allowlist: a script was tested only if someone
remembered to add it, so every new tutorial was uncovered from birth.
Coverage was 4/26 in HowToGalaxy, 6/40 in HowToLens and 10/15 in
HowToFit. That is how a public teaching notebook stayed broken in three
places with CI green (HowToGalaxy #56/#57) - no job had ever executed it.
The only backstop is PyAutoHeart's workspace-smoke, which runs weekly.
Coverage is now opt-OUT. run_smoke.py becomes a thin shim over
PyAutoHands' autohands/run_python.py - the same entry point Heart's
workspace-validation uses for run_scripts - so the PR gate and the
validation runner cannot drift apart, and both read one exclusion list
(config/build/no_run.yaml, which the notebook runner already honoured).
Discovery, exclusion and env resolution are no longer reimplemented here.
--report-dir is passed deliberately, not cosmetically: run_python.py only
propagates failures when a report was built, and without it the suite
runs to completion and always exits 0. It also switches execute_script
from abort-on-first-failure to record-and-continue, matching the old
runner's behaviour. A negative control confirms a deliberate failure
turns the suite red.
Executed-script counts, measured, not assumed:
HowToGalaxy 4 -> 26
HowToLens 6 -> 39 (+1 deliberately skipped)
HowToFit 10 -> 15
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>1 parent 10992ec commit 6eaf6ed
1 file changed
Lines changed: 0 additions & 26 deletions
This file was deleted.
0 commit comments