Skip to content

iterator gap fix - #11

Merged
cboulay merged 4 commits into
devfrom
10-nwbslicer-should-deal-with-timestamp-gaps
Jul 20, 2026
Merged

iterator gap fix#11
cboulay merged 4 commits into
devfrom
10-nwbslicer-should-deal-with-timestamp-gaps

Conversation

@kylmcgr

@kylmcgr kylmcgr commented Jun 9, 2026

Copy link
Copy Markdown
Contributor
  • NWBAxisArrayIterator now splits chunks that contain a gap.
  • NWBClockDrivenProducer now returns a CoordinateAxis with the true per-sample timestamps instead of a LinearAxis.

@kylmcgr
kylmcgr requested review from cboulay and Copilot June 9, 2026 21:36
@kylmcgr kylmcgr self-assigned this Jun 9, 2026
@kylmcgr kylmcgr linked an issue Jun 9, 2026 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves correctness when NWB streams have explicit timestamps that include gaps by preventing time axes from implicitly “flattening” over missing time. It updates both chunked iteration and clock-driven playback to preserve true per-sample time semantics across gaps.

Changes:

  • Add gap detection (find_gaps) and use it to split iterator chunks that span timestamp gaps.
  • Update NWBSlicer.read_by_time (and NWBClockDrivenProducer) to emit a CoordinateAxis with true per-sample timestamps when a requested window spans a gap.
  • Add comprehensive tests + new gappy/irregular NWB fixtures to validate behavior, including the gap_tol control.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/ezmsg/nwb/slicer.py Adds gap detection helper and switches read_by_time to CoordinateAxis when windows span gaps.
src/ezmsg/nwb/iterator.py Splits chunk messages at detected timestamp gaps; plumbs gap_tol through iterator settings and prefetch worker.
src/ezmsg/nwb/clockdriven.py Forwards gap_tol into slicer time-window reads so clock-driven playback preserves true timestamps across gaps.
tests/conftest.py Adds session-scoped fixtures that generate synthetic gappy and irregular timestamped NWB files for testing.
tests/test_slicer.py Adds slicer-level tests covering gap-free windows, gap-spanning windows, inside-gap empty windows, and irregular streams.
tests/test_iterator.py Adds iterator-level tests ensuring chunks are split at gaps without dropping/reordering samples; tests gap_tol behavior.
tests/test_clockdriven.py Adds clock-driven tests verifying empty chunks inside gaps, CoordinateAxis for gap-spanning windows, and multi-producer sync across gaps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/ezmsg/nwb/slicer.py
Comment on lines 485 to +489
out_data = info.dset[start_idx:stop_idx]
time_axis = template.axes["time"]
ts_window = np.asarray(ts_arr[start_idx:stop_idx])
has_gain = hasattr(time_axis, "gain")

@cboulay
cboulay force-pushed the 10-nwbslicer-should-deal-with-timestamp-gaps branch from 46b854c to a78f546 Compare July 20, 2026 15:35
@cboulay
cboulay merged commit 6f83a54 into dev Jul 20, 2026
14 checks passed
@cboulay
cboulay deleted the 10-nwbslicer-should-deal-with-timestamp-gaps branch July 20, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NWBSlicer should deal with timestamp gaps

3 participants