Skip to content

[feat] Add F1 score workflow in validation step#6

Open
gudgud96 wants to merge 4 commits into
masterfrom
feature/eval-f1-in-validation
Open

[feat] Add F1 score workflow in validation step#6
gudgud96 wants to merge 4 commits into
masterfrom
feature/eval-f1-in-validation

Conversation

@gudgud96
Copy link
Copy Markdown
Owner

All configs related to calculating F1 within validation step is under eval. Same set of config is used by test.py.

eval_first_n_examples: Only run F1 workflow for first N examples in validation set
eval_after_num_epoch: Only run F1 workflow after certain number of epochs
eval_per_epoch: Interval for running F1 workflow
exp_tag_name: Just insert some random name
audio_dir: Audio path for calculating F1 score
midi_dir: MIDI path for calculating F1 score

Also reduced num_workers for both train and validation to 4, as observed that 12 workers take long for process creation (especially after adding in F1 calculation workflow), but gives relatively minimal speedup.

** To use validation set for F1 calculation, please run midi_script.py to re-generate the MIDI files from stems for the validation set.

For training, refer to config/config_slakh_f1=0.65.yaml for the most up-to-date changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant