Skip to content

[feat] Change TF spectral ops to torchaudio#7

Open
gudgud96 wants to merge 3 commits into
masterfrom
refactor/remove-tf
Open

[feat] Change TF spectral ops to torchaudio#7
gudgud96 wants to merge 3 commits into
masterfrom
refactor/remove-tf

Conversation

@gudgud96
Copy link
Copy Markdown
Owner

@gudgud96 gudgud96 commented Nov 26, 2023

I faced some deadlock issue when running with more than 4 dataloader workers, analyzed and found that the issue is due to stuck in TF functions in spectrogram.py.

TF can't be entirely migrated unless we can rewrite vocabularies.py and migrate entirely from seqio and t5. So, the main changes are:

  1. Reduced num_workers, increased every_n_epochs for checkpointing, check_val_every_n_epoch for validation;
  2. Migrate spectrogram.py to use torchaudio
  3. Added use_tf_spectral_ops in dataset and during evaluation, to choose whether to use TF or torchaudio's melspectrogram (default to torchaudio)
  4. Removed redundant TF related code in vocabularies.py and metrics_utils.py

Other minor changes:

  1. Added eval flags for configs needed when running test.py
  2. Added split_frame_length in dataset config (2000 for current training, mel_length if wanted to ensure contiguous frames)
  3. Added is_deterministic, is_randomize_tokens in dataset config

@gudgud96 gudgud96 requested a review from KinWaiCheuk December 7, 2023 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant