Discrepancy between reproduce result and proposed result.

[Ben1001409]

Hi @wjq-learning , Thanks for your outstanding work. I have tried to reproduce your experiment using your code and the datasets specified, but i obtained a large discrepancy result between proposed result and reproduced results especially in SHHS1. All the experiment settings are configured same as the paper.

Here is the proposed result:

Here is the reproduced result:

Are there any different version of results or mistakes in the proposed result?. I would really appreciate your response.

[wjq-learning]

Hello, thank you for your interest in this work and for your effort in reproducing the results.

From the results you provided, it seems that the performance on datasets other than SHHS1 is generally within a reasonable range of variation. However, the discrepancy on SHHS1 might be caused by differences in hyperparameter settings, random seeds, or environmental factors.

I would suggest trying different hyperparameter configurations (e.g., learning rate, batch size, optimizer settings). These adjustments may help reduce the gap.

If you encounter any specific issues during the reproduction process, please feel free to let me know, and I’ll do my best to assist you.

Thank you again for your careful work!

[Ben1001409]

Thanks for your reply @wjq-learning, all the preprocessing steps i conduct is the same as the code you provided, and the batch size used is 32 and lr 0.001. Is the number of encoder layer in sequence encoder set to 1?