Skip to content

Conversation

@bendichter
Copy link
Contributor

@bendichter bendichter commented Oct 25, 2025

Add comprehensive support for audio and video recordings in behavioral
experiments:

- Add audio file extensions (mp3, wav) and video file extensions
  (mp4, mkv, avi) with corresponding _audio and _video suffixes
- Document usage of audio/video recordings in beh directory for
  capturing vocalizations, speech, facial expressions, and body movements
- Add metadata schema for audio/video device information and stream
  properties
- Include privacy warnings about personally identifiable information
  in human subject recordings
- Update behavioral experiments title to remove "with no neural
  recordings" restriction, clarifying data can be stored with or
  without neural recordings
- Add examples for file organization including multi-angle recordings
  and split files
- Define optional entities: task, acquisition, run, recording, split
@yarikoptic yarikoptic changed the title SCHEMA: Add audio video SCHEMA: Add audio video behavioral data support Oct 25, 2025
@yarikoptic yarikoptic added the schema Issues related to the YAML schema representation of the specification. Patch version release. label Oct 25, 2025
…ee macros

- Change section title from 'Behavioral experiments' to 'Behavioral recordings'
- Convert file tree examples to use MACROS___make_filetree_example for consistent rendering
- Address review comments from @yarikoptic in PR #2231
Copy link
Collaborator

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this makes sense to me. It would be good to get some feedback from contributors to related BEPs, such as eye-tracking (20), motion (29), stimuli (44) and physio (45). Even if this PR doesn't propose adding this as an associated file to those data types, the potential is there and it's worth getting opinions and identifying potential conflicts.

cc @bids-standard/bep029 @bids-standard/bep044
cc @mszinte @julia-pfarr @oesteban (BEP020)
cc @m-miedema @smoia @SouravKulkarni (?) (BEP045)

@effigies effigies changed the title SCHEMA: Add audio video behavioral data support [ENH] Add audio/video recordings to behavioral experiments Oct 28, 2025
@codecov
Copy link

codecov bot commented Oct 28, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.86%. Comparing base (7c06228) to head (aead0a4).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2231   +/-   ##
=======================================
  Coverage   82.86%   82.86%           
=======================================
  Files          20       20           
  Lines        1669     1669           
=======================================
  Hits         1383     1383           
  Misses        286      286           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Collaborator

@robertoostenveld robertoostenveld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me, I look forward to having this as part of the specification

@satra
Copy link
Collaborator

satra commented Dec 3, 2025

a colleague asked why all recording sources are being put into a single folder when other parts of bids are separated by modality/technique. perhaps a sentence or two for this rationale would be helpful to include in the specification.

@bendichter
Copy link
Contributor Author

bendichter commented Dec 3, 2025

@satra The rationale here is that the organizing principle for the beh/ directory is the type of data being captured (behavior) rather than the recording technique (audio, video, events, etc.).

In the context of behavioral experiments, audio recordings (capturing vocalizations, speech), video recordings (capturing facial expressions, body movements), and event files all serve the same fundamental purpose: documenting subject behavior. They are different modalities of the same underlying phenomenon.

This is analogous to how func/ contains both BOLD and ASL data - different acquisition techniques measuring the same underlying phenomenon (functional brain activity).

Additionally, keeping these files together in beh/ makes it easier to:

  1. Find all behavioral data for a session in one place
  2. Maintain temporal alignment metadata between synchronized recordings
  3. Keep behavioral data organized when collected alongside neural recordings (where the neural data lives in its respective modality folder like ieeg/ or eeg/)

I can add a sentence or two to the specification explaining this rationale. Would something like the following work?

"Audio and video recordings are stored together in the beh/ directory because they represent different modalities for capturing the same underlying phenomenon: subject behavior. This organization mirrors other BIDS directories where the organizing principle is the type of data being measured rather than the specific recording technique."

@satra did I understand the question? Do you think this would satisfy the colleague in the context of your conversation with them?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

schema Issues related to the YAML schema representation of the specification. Patch version release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BEP for audio/video capture of behaving subjects

7 participants