The difference between the reproduced results and that in the paper on the dataset BCIC-2A

[ejejeieei]

Dear author, thanks for your excellent work. But when I reproduced experiment, I found the result is different from the result in the paper by using provided code in this repo. Is there any problem in my experiment? Here are my experiment background and the result I made.
used file : linear_probe_EEGPT_BCIC2A.py
GPU: 4090 24GB
torch: 2.3.1 + cu11.8

[gsavelev]

@ejejeieei, could you please show me the source (page number / repo path) of published results of evaluating EEGPT on BCIC-2A? Did not find it anywhere.

[ejejeieei]

@gsavelev of course, the result is on the Page 8 of the paper

[gsavelev]

@ejejeieei, on arxiv only MIBCI and BCIC4-1 in Motor Imagery eval section. Do you have another version of paper?

[ejejeieei]

The result was shown in Openreview version，there is the link:https://openreview.net/forum?id=lvS2b8CjG5

[Daniel-Lin-S]

I noticed that the authors did not clearly explain the meaning of [value] +- [value] in tables 2-4. But observing the extremely small second values, it may not be the standard deviation across subjects. Rather it looks like the standard deviation of "average accuracy across subjects" for different seeds. Though this observation does not change the fact that the average accuracy stated in the paper cannot be reproduced.

I am getting similar results on BCIC-2V dataset as you. I have changed parameters (div_factor, final_div_factor) passed to OneCycleLR to match the settings stated in appendix C.2.1 and all other settings are the same as in the original paper. (starting learning rate 5e-4, maximum 1e-3, minimum 4.22e-7)