Publish gates: swallow only time-empty results, pass channel-empty through - #180
Merged
Conversation
…rough Units that suppress empty publishes (Downsample, BinnedAggregate, Window) or use emptiness to terminate a drain loop (ResampleUnit, ResampleConcat) keyed on data.size == 0, which conflates two cases: no new samples along the time-like axis (correctly suppressed) vs. all channels/features sliced away upstream while time samples remain (e.g. Slicer with on_empty='warn'). The latter should flow downstream so consumers that align or merge multiple sources keep the stream's cadence. Add is_empty_along(msg, dims) to util.message and re-key each gate on the unit's time-like axis (settings.axis; for Window also the 'win' newaxis).
ResampleProcessor.__next__ signals 'not initialized' with a dimensionless null template (dims=[''], key='null') that has no time axis at all, so the is_empty_along gate published it -- and kept draining -- flooding downstream with null messages until the first reference arrived (test_resample_system_reference_driven caught this). Add has_samples_along(msg, dim) -- dim present AND nonzero -- and use it in both drain loops: anything that is not a well-formed chunk along the resample axis means 'nothing ready'. Channel-empty chunks still publish.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Units that suppress empty publishes —
Downsample,BinnedAggregate,Window— or use emptiness to terminate a drain loop —ResampleUnit,ResampleConcat— keyed ondata.size == 0. That conflates two cases:Slicerwithon_empty="warn"(Slicer: add on_empty setting — no-match selections warn and emit empty by default #177) sliced away every channel while time samples remain — which should flow downstream so consumers that align or merge multiple sources keep the stream's cadence.Changes
is_empty_along(message, dims)inezmsg.sigproc.util.message: true iff any named dim is present in the message with zero length (absent dims ignored).Downsample.on_signal/BinnedAggregate.on_signal: swallow only whensettings.axis(default"time") is 0-length.Window.on_signal: swallow only when thewinnewaxis (or, in pass-through mode, the windowed axis) is 0-length.ResampleUnitpublisher /ResampleConcat._drain: terminate the drain only on an empty resample axis. This one was the sharpest edge: with a 0-channel stream, every chunk hadprod(shape) == 0, so the drain would break before publishing and silently drop real time chunks indefinitely.Not changed (already correct)
The remaining
size == 0sites (filter.py,butterworthzerophase.py,fbcca.py,rollingscaler.py,filterbank.py) are compute guards that pass the message through unchanged rather than swallowing it — passthrough is already right for a channel-empty message.ResampleConcatProcessor.__next__'sshape[0] == 0check was already time-axis-based.Testing
tests/unit/test_empty_gate.py(5 tests): the helper;Downsample,BinnedAggregate, andWindowdriven directly throughon_signal, asserting channel-empty results publish and time-empty results are swallowed; and a stub-processor test provingResampleConcat._drainno longer terminates on a 0-feature chunk.