Pangu lite training

[mikael10j]

Hello,

I'm trying to reproduce the Pangu Lite results shown in the README. However, although I followed the pseudocode and the paper for the implementation, I don't get the same RMSEs (I'm quite far actually).

Could you please confirm if you used the same hyperparameters as for the full version:
optimizer = Adam
lr = 5e-4
weight decay = 3e-6
lr scheduler = cosine

Thank you