Problem when Pretrainig the model with past_feat_dynamic_real

[bkoyuncu]

Discussed in #169

^{Originally posted by bkoyuncu January 8, 2025}
Hello,

I have created my data with past_feat_dynamic_real, however, I believe the current implementation in pretrain.py does not include them in training as:

uni2ts/src/uni2ts/model/moirai/pretrain.py

Line 485 in cadebd8

+ SelectFields(fields=list(self.seq_fields))

where self.seq_fields are defined as

uni2ts/src/uni2ts/model/moirai/pretrain.py

Lines 66 to 72 in cadebd8

	seq_fields: tuple[str, ...] = (
	"target",
	"observed_mask",
	"time_id",
	"variate_id",
	"prediction_mask",
	"patch_size",

I believe past_feat_dynamic_real are not being output in the end of processing the input data.

Is this an expected behaviour? How can I make sure past_feat_dynamic_real are used in training?

[bkoyuncu]

I am thinking if past_feat_dynamic_real needs to be truncated with other inputs as these are known context features, which should not be masked at all.

uni2ts/src/uni2ts/model/moirai/pretrain.py

Lines 447 to 454 in cadebd8

	+ MaskedPrediction(
	min_mask_ratio=self.hparams.min_mask_ratio,
	max_mask_ratio=self.hparams.max_mask_ratio,
	target_field="target",
	truncate_fields=("variate_id", "time_id", "observed_mask"),
	optional_truncate_fields=("past_feat_dynamic_real",),
	prediction_mask_field="prediction_mask",
	expected_ndim=3,

[liuxu77]

Hi @bkoyuncu, I think past_feat_dynamic_real should be involved in the "target", and at the same time, their prediction_mask are all set to False.