Skip to content

BinnedAggregate: allow a tuple of operations, stacked on a metric axis - #193

Merged
cboulay merged 2 commits into
devfrom
feat/binned-aggregate-multi-op
Aug 4, 2026
Merged

BinnedAggregate: allow a tuple of operations, stacked on a metric axis#193
cboulay merged 2 commits into
devfrom
feat/binned-aggregate-multi-op

Conversation

@cboulay

@cboulay cboulay commented Aug 4, 2026

Copy link
Copy Markdown
Member

Lets BinnedAggregateSettings.operation be a tuple, applying several aggregations to the same bins and stacking the results on a trailing axis.

First of a stack of two. #194 builds on this branch.

Why

Putting a high-rate signal on a screen wants each bin's min and max. Stride decimation lands between samples and clips the peaks, so a spike shrinks or vanishes depending on where the stride falls; the two extremes preserve peak amplitude exactly. That was going to be a dedicated min/max decimator until it became clear the only thing it would do that BinnedAggregate does not is apply two functions instead of one — everything else (binning on the shared BinSchedule, carrying the open partial bin across message boundaries, labelling the output axis) already lives here and is better tested than a new transformer would have been.

What

BinnedAggregate(BinnedAggregateSettings(
    bin_duration=0.001,
    operation=(AggregationFunction.MIN, AggregationFunction.MAX),
))
# (time, ch) @ 30 kHz  ->  (time, ch, metric) @ 1 kHz

The trailing axis is named by newaxis (default "metric") and coordinate-labelled with the AggregationFunction values — "min", "max" — so consumers read names rather than relying on position.

Design notes

  • Trailing, not in place, so the binned axis keeps its position.
  • Output shape follows the type of operation, not the count. A scalar behaves exactly as before — no trailing axis, no new key in axes — so existing streams are untouched. A one-element tuple does produce the axis, so a caller assembling its tuple programmatically gets a stable shape either way.
  • Computing them together is not just less code: the bins are cut once and sliced once, so the results are identical by construction rather than by two transformers agreeing on a grid. A test compares a tuple against separate single-op transformers over the same chunking to hold that.
  • The empty path needed the axis at full width too. At a high input rate most chunks close no bin, so _empty_like is the common path, and a zero-length message of the wrong rank will not concatenate with the messages around it.
  • The metric axis depends only on settings, so it is built in _reset_state and held in state; every output carries the same object, which keeps a downstream identity check cheap.

Tests

10 new, on top of the existing 20. 98 pass across test_binned_aggregate, test_aggregate, test_bin_schedule, plus test_empty_gate (the only other BinnedAggregate consumer).

cboulay added 2 commits August 3, 2026 22:26
Reducing a high-rate signal for display wants each bin's min *and* max --
stride decimation lands between samples and clips the peaks, so a spike
shrinks or vanishes depending on where the stride falls, whereas the two
extremes preserve peak amplitude exactly. That is the only thing a
dedicated min/max decimator would do that this transformer did not
already do: bin on a shared schedule, carry the open partial bin across
message boundaries, aggregate, label the output axis. Rather than write a
second transformer with its own copy of that arithmetic, `operation` now
accepts a tuple.

Each function is applied to the same bins and the results stack on a
trailing axis, `newaxis` (default "metric"), coordinate-labelled with the
AggregationFunction values -- "min", "max" -- so a consumer reads names
rather than relying on positional convention. Trailing rather than in
place, so the binned axis keeps its position.

Computing them together is not just less code: the bins are cut once and
sliced once, so the results are identical by construction rather than by
two transformers agreeing. There is a test comparing a tuple against
separate single-op transformers over the same chunking to hold that.

The output shape follows the *type* of `operation`, not the count. A
scalar behaves exactly as before -- no trailing axis, no new key in
`axes` -- so existing streams are untouched, and a one-element tuple does
produce the axis, so a caller assembling its tuple programmatically gets
a stable shape either way.

The empty-payload path has to build the trailing axis at full width too;
a zero-length message of the wrong rank will not concatenate with the
messages around it, and at a high input rate most chunks close no bin, so
that path is the common one rather than an edge case.
It depends only on settings -- the operations and the axis name -- so
rebuilding it in _out_axes on every message meant constructing a numpy
string array per message on the hot path, for a value that cannot have
changed.

Moved to _reset_state and held in state. Every output now carries the
same object, so a downstream identity check on the axis is a pointer
comparison rather than an array comparison.
@cboulay
cboulay merged commit bcbcbee into dev Aug 4, 2026
14 checks passed
@cboulay
cboulay deleted the feat/binned-aggregate-multi-op branch August 4, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant