How to totally freeze the grad?

[Adolfhill]

Hello, thanks for your impressive work!
I am currently attempting to train LightGLUE with SILK. However, I've noticed that using SILK for extracting key points requires significantly more time and graphic memory compared to SuperPoint.
Have you ever tried to do the same thing or encountered this issue? Is that normal?
I am concerned that I might not have properly frozen some gradients.
Below is the code snippet I used to freeze the gradients. Is this the correct approach, or should I do something else?

def get_model(
    checkpoint,
    nms,
    device,
    default_outputs,
    backbone,
    detection_threshold,
    detection_top_k,
    border_dist,
    descriptor_scale_factor
):
    # load model
    model = SiLK(
        in_channels=1,
        backbone=deepcopy(backbone),
        detection_threshold=detection_threshold,
        detection_top_k=detection_top_k,
        nms_dist=nms,
        border_dist=border_dist,
        default_outputs=default_outputs,
        descriptor_scale_factor=descriptor_scale_factor,
        padding=0,
    )
    model = load_model_from_checkpoint(
        model,
        checkpoint_path=checkpoint,
        state_dict_fn=lambda x: {
            k[len("_mods.model."):]: v for k, v in x.items()},
        device=device,
        freeze=True,
        eval=True,
    )
    return model