Shuffling data samples during training

[vinh-cao]

Hi Jan,

may I ask you a question: why don't you shuffle your training data? As far as I know, shuffling a batch helps the model to converge fast and it prevents any bias during the training. Since you only use the encoder and slice fixed data samples, there is no temporal relationship between the samples anymore.
Can you share a little more insight about your training configuration of 1k stocks? I am struggling with the training, since most of the experiments result in straight line or very poor performance.

Have you tried the Spacetimeformer? It is a great idea to have cross attention on multivariate architecture.

Thanks for sharing your great works.
Best,
Vinh

If you are not shuffling ur files during training, it looks like that the last files that go into the generator have a lot of entries. What I can deriving from shape[3736448,256] is that ur are passing 3736448 sequences with a length of 256 into the model.

The 3736448 is the aggregated batch size of that file batch.

Just check whether u have very large file in ur dataset and potentially exclude it for now.

Originally posted by @JanSchm in #1 (comment)