The results using the checkpoint for stage1 do not match expectations.

[pxy963]

Hello,
I use your checkpoint to experiment on the retrieval for swiss prot and the proteinkg25. But I follow the guide in the readme, and I get the result is lower about 5% in the testset of the two datasets than the point in the paper. Is the checkpoint you provided the different from the results in the paper, or are there some experimental settings involved?
Thank you.

[acharkq]

Hi, can you provide more details here?

[pxy963]

I run this command
'python stage1.py --devices '0,1,2,3' --mode eval --filename stage1_ckpt --num_query_token 8 --plm_name "facebook/esm2_t30_150M_UR50D" --save_every_n_epochs 10 --batch_size 32 --precision 'bf16-mixed' --num_workers 8 --init_checkpoint /path/to/stage1.ckpt;'
and I download your "stage1.ckpt" to test.
For example, when retrieval in the test set of the swissprot, the T2P R@20 is 90.7, but in the paper is 95.8.