[feat] Change TF spectral ops to torchaudio#7
Open
gudgud96 wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I faced some deadlock issue when running with more than 4 dataloader workers, analyzed and found that the issue is due to stuck in TF functions in
spectrogram.py.TF can't be entirely migrated unless we can rewrite
vocabularies.pyand migrate entirely fromseqioandt5. So, the main changes are:num_workers, increasedevery_n_epochsfor checkpointing,check_val_every_n_epochfor validation;spectrogram.pyto use torchaudiouse_tf_spectral_opsin dataset and during evaluation, to choose whether to use TF or torchaudio's melspectrogram (default to torchaudio)vocabularies.pyandmetrics_utils.pyOther minor changes:
evalflags for configs needed when runningtest.pysplit_frame_lengthin dataset config (2000 for current training,mel_lengthif wanted to ensure contiguous frames)is_deterministic,is_randomize_tokensin dataset config