fix: make script_timing baselines real — run-identity dedup + rename-aware slugs (#165) - #166
Merged
Merged
Conversation
…rename-aware slugs
Two defects made every stored script-timing baseline unusable (PyAutoHeart#165).
1. The window filled with copies of ONE run. The tick re-reads
run_logs/latest every cycle and update_history appended those same
durations each time, so a window of seven held seven copies of a single
observation: median(prior) was that observation, stable by construction,
and one unlucky run read as a regression.
History entries now carry provenance — {"duration_s", "run_id", "ts"} —
where run_id is the real name of the timestamped run dir behind the
`latest` symlink. Re-observing the run already at the head of the window
REPLACES that entry instead of appending, so ticks are idempotent and
seven entries mean seven distinct runs. Legacy bare-float histories still
read (run_id ""); one whose values are all identical is provably that
same artefact and collapses to a single entry on first touch.
Classification now waits for a real baseline: fewer than three distinct
prior runs counts the script under a new `building_count` rather than
green/yellow/red, so a thin baseline never masquerades as a verdict.
red/yellow/green_count keep their exact meaning for the dashboard.
2. Moving a script orphaned its history. The slug is path-derived, so a
directory restructure stranded the baseline under a name nothing writes
to again — and the check then had nothing to compare against and silently
never fired. Every scan now notices history files it did not touch: where
exactly one such orphan matches a script that has no history (same
workspace, same leaf script name) its history is renamed onto the new
slug before this run is appended (`migrated_count` / `migrated`).
Ambiguous or unmatched orphans are never deleted or guessed at — they are
reported as `orphaned_count` / `orphaned` and in the one-line output, so
the silent case becomes a loud one.
History writes now go through heart.state.atomic_write_json (they were bare
write_text, against the house rule).
Tests: 6 -> 16 in tests/test_script_timing.py, covering same-run re-tick,
distinct-run accumulation to the cap, the legacy collapse, legacy mixed
histories still classifying, the three-distinct-run floor, unambiguous and
ambiguous migration, the on-disk entry shape and the atomic write. Four
existing tests asserted the old behaviour (repeated observation of one
results dir standing in for repeated runs) and were rewritten to use
distinct run dirs; workspace names in the suite are now fictional, per the
tenant firewall. Full suite green (583 passed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoDPz2LevKeBaDwqFKtZrU
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #165. Phase 0 of the test-performance board arc (#163): the two documented defects that made the per-script baselines unusable, fixed at the source.
run_logs/latestand re-appended the same observation until the window filled with copies of one run. History entries now carry{duration_s, run_id, ts}(run_id = the resolved timestamped run dir behind thelatestsymlink); a re-tick on the same run replaces the newest entry instead of appending — the window only grows across distinct runs. Legacy bare-float histories still read; a legacy window of all-identical values collapses to the single observation it provably was.building_countinstead of being judged on a fake-stable median.red/yellow/green_countkeep their exact meaning —dashboard.pyuntouched.migrated. Ambiguous or unmatched orphans are reported (orphaned_count), never deleted, never silent.atomic_write_json(was a barewrite_text).Tests: 583 passed (+10;
test_script_timing.py6 → 16). Four existing tests had encoded the bug itself — re-running on one results dir as a stand-in for repeated runs — and now use genuinely distinct runs. Tenant firewall green.🤖 Generated with Claude Code
https://claude.ai/code/session_01EoDPz2LevKeBaDwqFKtZrU
Generated by Claude Code