Skip to content

fix: park EP script as NEEDS_FIX no_run for release testing - #131

Merged
Jammy2211 merged 1 commit into
mainfrom
fix/ep-no-run-release
Aug 3, 2026
Merged

fix: park EP script as NEEDS_FIX no_run for release testing#131
Jammy2211 merged 1 commit into
mainfrom
fix/ep-no-run-release

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Why

EP is not release-ready, so every EP script in the release matrix is being
parked. This is the autofit_workspace half; the root-cause investigation and
the failing script live in
autofit_workspace_test#82 (scripts/graphical/ep.py).

scripts/features/expectation_propagation.py passes today. It is parked
because it shares the EP machinery whose message projection is unstable, not
because it is currently failing.

Root cause (summary — full detail in autofit_workspace_test#82)

The 2026-08-03 nightly release integrate failed on graphical/ep.py
(PyAutoHeart run 30788224561). Reproduced locally under the release profile:

  • A truncated per-factor search projects a posterior with an effective sample
    size of 1
    — a single sample carries all the weight, so the weighted std of
    every parameter is exactly 0.
  • EP feeds that degenerate projection back as the next cycle's prior (q*
    arrives already collapsed: q*_sigma=1.5e-06 against a healthy
    cavity_sigma=30).
  • The next search draws initial points that all yield an identical log
    likelihood → InitializerException.

Damping does not help (q* is degenerate before damping applies) and
check_sigma_collapse misses it (the collapse is a single-step drop, not the
monotone shrinkage it looks for). See PyAutoFit #1332 F10.

Change

One NEEDS_FIX-tagged entry in config/build/no_run.yaml, so every mega-run
surfaces it with a warning banner until EP is fixed.

Verification

  • Pattern is .py-anchored and checked against the runner's own should_skip:
    exactly 1 script newly skipped, no over-match, not inert.
  • Parses via find_needs_fix_skips with the correct date and reason.
  • Verified by pattern-match rather than a full shard run (the shard is slow);
    the equivalent end-to-end runner check was done in autofit_workspace_test#82,
    where it exits 0.

🤖 Generated with Claude Code

EP is not release-ready. The per-factor searches in the EP scripts are
truncated (autofit_workspace_test graphical/ep.py caps maxcall/maxiter at
1000), so each search projects a posterior with an effective sample size of
1 - a single sample carries all the weight and the weighted variance of
every parameter is exactly 0. EP feeds that degenerate projection back as
the next cycle's prior, which is then either a delta function or, after the
cavity division cancels two near-identical natural-parameter vectors, a
prior whose mean has diverged (observed: shared centre = 6.4e7 for data
spanning ~100 pixels). Either way the next search draws initial points that
all yield an identical log likelihood, and PyAutoFit raises
InitializerException.

This is intermittent - it depends on where the truncated search happens to
stop - and it broke the 2026-08-03 nightly release integrate
(PyAutoHeart run 30788224561) after passing the five previous runs.

Damping does not fix it: q* arrives already collapsed, so an EP updater
with delta < 1 still ingests a zero-variance projection. The existing
check_sigma_collapse diagnostic also misses it, because the collapse is a
single-step drop rather than the monotone shrinkage it looks for.

Park every EP script in the release matrix rather than papering over the
instability in one script's search settings. Tagged NEEDS_FIX so each
mega-run surfaces them until the underlying EP projection is fixed
(PyAutoFit #1332 F10).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Jammy2211
Jammy2211 merged commit 34a4791 into main Aug 3, 2026
5 checks passed
@Jammy2211
Jammy2211 deleted the fix/ep-no-run-release branch August 25, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant