`NWBSlicer` should deal with timestamp gaps

[cboulay]

NWBClockDrivenProducer calls NWBSlicer's read_by_time. Here's the code in question:

ezmsg-nwb/src/ezmsg/nwb/slicer.py

Lines 454 to 480 in 46ef721

	# Materialize if h5py dataset for searchsorted
	if hasattr(timestamps, "shape") and not isinstance(timestamps, np.ndarray):
	ts_arr = timestamps[:]
	else:
	ts_arr = timestamps
	start_idx = int(np.searchsorted(ts_arr, t_start, side="left"))
	stop_idx = int(np.searchsorted(ts_arr, t_end, side="left"))
	out_data = info.dset[start_idx:stop_idx]
	if start_idx < len(ts_arr):
	chunk_t0 = ts_arr[start_idx]
	else:
	chunk_t0 = template.axes["time"].gain * start_idx if hasattr(template.axes["time"], "gain") else 0.0
	return replace(
	template,
	data=out_data,
	axes={
	**template.axes,
	"time": replace(
	template.axes["time"],
	offset=self._ts_off + chunk_t0,
	),
	},
	key=stream_key,
	)

The output time axis is always a LinearAxis with an offset and a gain, but it does not provide individual timestamps.

If the requested time range happens to span a chunk of data where samples are missing, and we would only know that if the NWB recorded all timestamps, then the returned chunk would not be able to differentiate between a chunk with samples missing in the middle or at the end. All the consumer would know was the first timestamp in the chunk and that the number of returned samples was less than expected.

Unfortunately, there isn't a simple way to fix this. The simplest that I can think of is:

• revise read_by_time return type to tuple[AxisArray, float] where the second value is ts_next: the timestamp associated with the next sample after the returned AxisArray.
• read_by_time will detect when the stop_idx is less than expected -- indicating missing samples -- and will truncate the returned data to stop before any sample gap.

The caller can use the ts_next as the first argument to the next call to read_by_time. This will ensure that we do not have individual AxisArray messages that span time gaps. But that's tricky because it will be overriding its input clock_tick which is intended to synchronize streams from different sources, so we'd be defeating its purpose.

I'm not sure what to do here...

[cboulay]

For NWBAxisArrayIterator, which is not clock-driven, I think this might be easier because we build chunk messages, plural, and enqueue them. It should be easy to simply split large chunks that span gaps into multiple chunks and enqueue the multitude.