-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DRAFT] Redesign/datasets ICA addition #56
base: redesign/datasets
Are you sure you want to change the base?
Conversation
…oper caching usage. ica still not leveraging caching correctly
Move this to the other PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this Victor! Here are my recommendations
@@ -95,10 +97,12 @@ def __init__( | |||
data, | |||
preload=False, | |||
verbose=None, | |||
ica_processor: Optional[ICAProcessor] = None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not a scalable way to integrate the ica processor, because if we were to continue like this, then everytime we want to add a new kind of processor, the lists of args would grow, and the class would have to be modified. We instead want to write code which is closed to modification, but open to extension. This means that we won't have to worry about breaking existing code, because we won't be modifying the existing code.
From what I can tell, there is no reason why the ICA processor cannot work as a regular dynamic item like the other preprocessing.
"""Compute ICA solution and save to disk.""" | ||
# High-pass filter for ICA | ||
raw_filtered = raw.copy() | ||
raw_filtered.filter(**self.filter_params) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make this optional since the data might already have been high-passed
bids_path.update( | ||
suffix="eeg", # override or confirm suffix | ||
extension=".fif", | ||
description="ica", # <-- This sets a desc=ica entity |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The desc should be something like ica-<hash of ica config>
to avoid cache issues
benchmarks/MOABB/dataio/ica.py
Outdated
ica_path = self.get_ica_path(raw_path) | ||
|
||
if not ica_path.exists(): | ||
ica = self.compute_ica(raw, ica_path) | ||
else: | ||
ica = mne.preprocessing.read_ica(ica_path, verbose="ERROR") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This caching introduces a bug... It only gets if any ica has been computed already for this raw file, but it does not check that the saved ica matches the parameters that we want
Co-authored-by: Drew Wagner <[email protected]>
added ica as dynamic item to preprocessing pipeline. uses BIDSPath and exposes method and other features. still has issues with the use of caching.