Check if without any code changes encoder models work with transformers backend or not. Should be trivial since encoder models does not use RoPe.
Check if without any code changes encoder models work with transformers backend or not. Should be trivial since encoder models does not use RoPe.