Add example code for loading Binary and Open Ephys data formats

Author: jsiegle

source/User-Manual/Data-formats/Binary-format.rst

@@ -29,7 +29,7 @@ Binary Format
 
 **Limitations**
 
-* Requires slightly more disk space because it stores two 64-bit timestamps for every sample.
+* Requires slightly more disk space because it stores one 64-bit sample number and one 64-bit timestamp for every sample.
 
 * Continuous files are not self-contained, i.e., you need to know the number of channels and the "bit-volts" multiplier in order to read them properly.
 
@@ -38,7 +38,9 @@ Binary Format
 File organization
 ####################
 
-Within a Record Node directory, data for each **experiments** (stop/start acquisition) is contained in its own sub-directory. Experiment directories are further sub-divided for individual **recordings** (stop/start recording).
+Within a Record Node directory, data for each **experiment** is contained in its own sub-directory. The experiment number is incremented whenever acquisition is stopped, which will reset the timestamps/sample numbers to zero. A new :code:`settings.xml` file will be created for every experiment.
+
+Experiment directories are further sub-divided for individual **recordings**. The recording number is incremented whenever recording is stopped, which will not reset the timestamps/sample numbers. Data from multiple recordings within the same experiment will have internally consistent timestamps/sample numbers, and will use the same :code:`settings.xml` file.
 
 .. image:: ../../_static/images/recordingdata/binary/organization.png
   :alt: Binary data directory structure
@@ -122,14 +124,72 @@ More detailed information about each electrode is stored in the :code:`structure
 Reading data in Python
 #######################
 
-* **(recommended)** Create a :code:`Session` object using the `open-ephys-python-tools <https://github.com/open-ephys/open-ephys-python-tools>`__ package. The data format will be automatically detected.
+Using :code:`open-ephys-python-tools`
+--------------------------------------
+
+The recommended method for loading data in Python is via the `open-ephys-python-tools <https://github.com/open-ephys/open-ephys-python-tools>`__ package, which can be installed via :code:`pip`.
+
+First, create a :code:`Session` object that points to the top-level data directory:
+
+.. code-block:: python
+
+   from open_ephys.analysis import Session
+
+   session = Session(r'C:\Users\example_user\Open Ephys\2025-08-03_10-21-22')
+
+The :code:`Session` object provides access to data inside each Record Node. If you only had one Record Node in your signal chain, you can find its recordings as follows:
+
+.. code-block:: python
+
+   recordings = session.recordnodes[0]
+
+:code:`recordings` is a list of :code:`Recording` objects, which is a flattened version of the original Record Node directory. For example, if you have three "experiments," each with two "recordings," there will be six total :code:`Recording` objects in this list. The one at index 0 will be recording 1 from experiment 1, index 1 will be recording 2 from experiment 1, etc.
+
+Each :code:`Recording` object provides access to the continuous data, events, spikes, and metadata for the associated recording. To read the continuous data samples for the first data stream, you can use the following code:
+ 
+.. code-block:: python
+
+   samples = recordings[0].continuous[0].get_samples(start_sample_index=0, end_sample_index=10000)
+   print(f'Stream name: {recordings[0].continuous[0].metadata("stream_name")}')
+
+This will automatically scale the data into microvolts before returning it.
+
+To read the events for the same recording:
+
+.. code-block:: python
+
+   events = recordings[0].events
+
+This will load the events for all data streams into a Pandas :code:`DataFrame` with the following columns:
+
+* :code:`line` - the TTL line on which this event occurred
+* :code:`sample_number` - the sample index at which this event occurred
+* :code:`timestamp` - the global timestamp (in seconds) at which this event occurred (defaults to -1 if all streams were not synchronized)
+* :code:`processor_id` - the ID of the processor from which this event originated
+* :code:`stream_index` - the index of the stream from which this event originated
+* :code:`stream_name` - the name of the stream from which this event originated
+* :code:`state` - 1 or 0, to indicate whether this event is associated with a rising edge (1) or falling edge (0)
+
+The fastest way to find the nearest continuous sample to a particular event is by using the :code:`np.searchsorted` function:
+
+.. code-block:: python
+
+   import numpy as np
+   event_index = 100 # the index of the event you're interested in
+   event_timestamp = events.iloc[event_index].timestamp 
+   nearest_continuous_sample_index = \
+      np.searchsorted(recordings[0].continuous[0].timestamps, 
+                      event_timestamp)
+
 
-* Create a :code:`File` object using the `pyopenephys <https://github.com/CINPLA/pyopenephys>`__ package.
+For more information on how to use the :code:`open-ephys-python-tools` library, check out this `README <https://github.com/open-ephys/open-ephys-python-tools/tree/main/src/open_ephys/analysis>`__
 
-* Use the :code:`DatLoad()` method from :code:`Binary.py` in the `open-ephys/analysis-tools <https://github.com/open-ephys/analysis-tools/blob/master/Python3/Binary.py>`__ repository.
+Using :code:`SpikeInterface`
+--------------------------------------
 
+You can also load data from the Open Ephys Binary format via `SpikeInterface <https://spikeinterface.readthedocs.io/en/stable/>`__, using the :code:`read_openephys()` method.
 
 Reading data in Matlab
 #######################
 
-* Use the `open-ephys-matlab-tools <https://github.com/open-ephys/open-ephys-matlab-tools>`__ library.
+Use the `open-ephys-matlab-tools <https://github.com/open-ephys/open-ephys-matlab-tools>`__ library, available via the `Matlab File Exchange <https://www.mathworks.com/matlabcentral/fileexchange/122372-open-ephys-matlab-tools>`__.

source/User-Manual/Data-formats/NWB-format.rst

@@ -106,6 +106,8 @@ Reading data in Python
 
 * Create a :code:`Session` object using the `open-ephys-python-tools <https://github.com/open-ephys/open-ephys-python-tools>`__ package. The data format will be automatically detected.
 
+* Alternatively, you can use the `PyNWB <https://pynwb.readthedocs.io/en/stable/>`__ package.
+
 
 Reading data in Matlab
 #######################

source/User-Manual/Data-formats/Open-Ephys-format.rst

@@ -118,14 +118,73 @@ Since the samples are saved as 16-bit unsigned integers, converting them to micr
 Reading data in Python
 #######################
 
-* **(recommended)** Create a :code:`Session` object using the `open-ephys-python-tools <https://github.com/open-ephys/open-ephys-python-tools>`__ package. The data format will be automatically detected.
 
-* Create a :code:`File` object using the `pyopenephys <https://github.com/CINPLA/pyopenephys>`__ package.
+Using :code:`open-ephys-python-tools`
+--------------------------------------
 
-* Use the :code:`loadContinuous`, :code:`loadEvents`, or :code:`loadSpikes` methods from :code:`OpenEphys.py` in the `open-ephys/analysis-tools <https://github.com/open-ephys/analysis-tools/blob/master/Python3/OpenEphys.py>`__ repository.
+The recommended method for loading data in Python is via the `open-ephys-python-tools <https://github.com/open-ephys/open-ephys-python-tools>`__ package, which can be installed via :code:`pip`.
 
+First, create a :code:`Session` object that points to the top-level data directory:
+
+.. code-block:: python
+
+   from open_ephys.analysis import Session
+
+   session = Session(r'C:\Users\example_user\Open Ephys\2025-08-03_10-21-22')
+
+The :code:`Session` object provides access to data inside each Record Node. If you only had one Record Node in your signal chain, you can find its recordings as follows:
+
+.. code-block:: python
+
+   recordings = session.recordnodes[0]
+
+:code:`recordings` is a list of :code:`Recording` objects, which is a flattened version of the original Record Node directory. For example, if you have three "experiments," each with two "recordings," there will be six total :code:`Recording` objects in this list. The one at index 0 will be recording 1 from experiment 1, index 1 will be recording 2 from experiment 1, etc.
+
+Each :code:`Recording` object provides access to the continuous data, events, spikes, and metadata for the associated recording. To read the continuous data samples for the first data stream, you need to first set the sample range (to prevent all samples from being loaded into memory), and then call :code:`get_samples()`:
+ 
+.. code-block:: python
+   
+   recordings[0].set_sample_range([10000, 50000])
+   samples = recordings[0].continuous[0].get_samples(start_sample_index=0, end_sample_index=10000)
+   print(f'Stream name: {recordings[0].continuous[0].metadata("stream_name")}')
+
+This will automatically scale the data into microvolts before returning it.
+
+To read the events for the same recording:
+
+.. code-block:: python
+
+   events = recordings[0].events
+
+This will load the events for all data streams into a Pandas :code:`DataFrame` with the following columns:
+
+* :code:`line` - the TTL line on which this event occurred
+* :code:`sample_number` - the sample index at which this event occurred
+* :code:`processor_id` - the ID of the processor from which this event originated
+* :code:`stream_index` - the index of the stream from which this event originated
+* :code:`stream_name` - the name of the stream from which this event originated
+* :code:`state` - 1 or 0, to indicate whether this event is associated with a rising edge (1) or falling edge (0)
+
+The fastest way to find the nearest continuous sample to a particular event is by using the :code:`np.searchsorted` function:
+
+.. code-block:: python
+
+   import numpy as np
+   event_index = 100 # the index of the event you're interested in
+   event_timestamp = events.iloc[event_index].timestamp 
+   nearest_continuous_sample_index = \
+      np.searchsorted(recordings[0].continuous[0].sample_numbers, 
+                      event_sample_number)
+
+
+For more information on how to use the :code:`open-ephys-python-tools` library, check out this `README <https://github.com/open-ephys/open-ephys-python-tools/tree/main/src/open_ephys/analysis>`__
+
+Using :code:`SpikeInterface`
+--------------------------------------
+
+You can also load data from the Open Ephys Binary format via `SpikeInterface <https://spikeinterface.readthedocs.io/en/stable/>`__, using the :code:`read_openephys()` method.
 
 Reading data in Matlab
 #######################
 
-* Use the `open-ephys-matlab-tools <https://github.com/open-ephys/open-ephys-matlab-tools>`__ library.
+Use the `open-ephys-matlab-tools <https://github.com/open-ephys/open-ephys-matlab-tools>`__ library, available via the `Matlab File Exchange <https://www.mathworks.com/matlabcentral/fileexchange/122372-open-ephys-matlab-tools>`__.