Skip to content

Own axes retained in processor state; Flatten variable-chunk hash; RangedAggregate coordinate-axis fixes - #181

Merged
cboulay merged 3 commits into
devfrom
owned-cached-axes
Jul 22, 2026
Merged

Own axes retained in processor state; Flatten variable-chunk hash; RangedAggregate coordinate-axis fixes#181
cboulay merged 3 commits into
devfrom
owned-cached-axes

Conversation

@cboulay

@cboulay cboulay commented Jul 22, 2026

Copy link
Copy Markdown
Member

Summary

Input axes may be views into an ezmsg transport buffer whose lifetime ends after the current subscriber callback, so any axis object a processor retains across calls must own its memory. This PR audits every _reset_state/template-caching site in the package and fixes the ones that aliased input axes, plus two adjacent bugs found during the audit.

Axes ownership (deep-copy retained axes)

  • HybridAxisArrayBuffer: _initialize deep-copies the non-target axes kept in the output template message.
  • ConcatProcessor: _build_cached_axes deep-copies cached pass-through axes.
  • FilterbankTransformer: template axes were a shallow dict copy that still shared the axis objects; now deep-copied.
  • CWTTransformer: template axes now deep-copied.
  • RangedAggregateTransformer: ax_vec stored a reference to the input coordinate axis data and read it on later messages (trapezoid x-coordinates, argmin/argmax lookups); now copied.

Audited-clean (no change needed): slicer and affinetransform already copy via np.array(); fir_hilbert copies its delay buffer; window/spectrum/adaptive_lattice_notch cache only scalar-field or freshly-allocated axes; remaining _reset_state implementations cache no message-derived axis objects.

Flatten: variable preserve-axis chunk sizes

_hash_message excluded the preserve-axis length so variable-length time chunks no longer reset state and rebuild the merged output axis; reshape takes the preserve-axis length from each live message.

RangedAggregate: CoordinateAxis band labels were broken

The numeric branch raised NameError (sl_dat unbound) on the first band, and the string branch indexed ax_vec with the band values instead of the matched indices. Labels are now "first - last" of the matched entries.

Tests

  • New tests assert outputs reuse one processor-owned axis object rather than aliasing input axes (buffer, concat, filterbank, CWT, ranged-aggregate), including a simulated transport-buffer-reuse case that overwrites the first message's axis memory.
  • New coverage for Flatten with varying time-chunk sizes and for RangedAggregate string-label coordinate axes.
  • Full unit suite: 3617 passed, 5 skipped.

cboulay added 3 commits July 21, 2026 23:00
Variable-length chunks along the preserve axis (e.g. irregular time
chunks) previously reset FlattenTransformer state and rebuilt the merged
output axis on every size change. Hash only the non-preserve shape and
take the preserve-axis length from each live message when reshaping.
Input axes may be views into an ezmsg transport buffer whose lifetime
ends after the current subscriber callback. Any axis object retained in
processor state beyond that call must own its memory:

- HybridAxisArrayBuffer._initialize: deepcopy the non-target axes kept
  in the output template message.
- ConcatProcessor._build_cached_axes: deepcopy cached pass-through axes.
- FilterbankTransformer._reset_state: deepcopy template axes (the
  shallow dict copy still shared the axis objects).
- CWTTransformer._reset_state: deepcopy template axes.

Tests assert outputs reuse one processor-owned axis object rather than
aliasing input axes.
_reset_state stored a reference to the input coordinate axis data, and
_process reads it on later messages (trapezoid x-coordinates,
argmin/argmax lookups); copy it into processor-owned memory since the
input axis may be a view into an ezmsg transport buffer.

The CoordinateAxis output-label paths were also broken: the numeric
branch appended the band mean and then hit ax_dat.append(sl_dat) with
sl_dat unbound (NameError on the first band), and the string branch
indexed ax_vec with the band values instead of the matched indices.
Labels are now 'first - last' of the matched entries.
@cboulay
cboulay merged commit 2d3387f into dev Jul 22, 2026
14 checks passed
@cboulay
cboulay deleted the owned-cached-axes branch July 22, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant