finetune with fixed context_length and pred_length

[NaNYeYuan]

I am trying to finetune with fixed context_length and pred_length via loading train data with SimpleEvalDatasetBuilder.
However, the eval prediction result is extraordinarily large.
What's right way to finetune with fixed context_length and pred_length?

[NaNYeYuan]

By the way, how to get prediction result from a raw sequence list?

[chenghaoliu89]

Hi @NaNYeYuan sorry for the late response.

The default_train_transfrom define the data processing pipeline. If you want fixed context_length and pred_length, I would suggest you to modify MaskedPrediction Class, which originally randomly sample the prediction length and context length.

It seems to be a common feature for model fine-tuning, we will implement the FixedMaskedPrediction in the future.