Thanks for sharing this great work! While reproducing the code, I ran into two issues:
1. Missing .prev_sample
In trainer_far_wan_anyflow_onpolicy.py:120, .prev_sample appears to be missing from the scheduler step output.
2. Training instability
Training the student with train_farwan1b_student_shift5_81f_480p_lr5e-5_6k_b32.yml, grad_norm is large (~150) on the first iteration and losses diverge after ~100 steps.
Could you take a look when you have a chance? Happy to share more details from my runs. Thanks!
Thanks for sharing this great work! While reproducing the code, I ran into two issues:
1. Missing
.prev_sampleIn
trainer_far_wan_anyflow_onpolicy.py:120,.prev_sampleappears to be missing from the scheduler step output.2. Training instability
Training the student with
train_farwan1b_student_shift5_81f_480p_lr5e-5_6k_b32.yml,grad_normis large (~150) on the first iteration and losses diverge after ~100 steps.Could you take a look when you have a chance? Happy to share more details from my runs. Thanks!