How to test the trained model of .ckpt format

[yihp]

Hi! Thanks for your contribution. It is an excellent piece of work!

The trained model is saved in last.ckpt format, but it cannot be directly tested using the test command in the repository.

dlhpcstarter -t cxrmate_hf -c config/test_huggingface/longitudinal_gen_prompt_cxr-bert.yaml --stages_module tools.stages --test

What adjustments should I make?

Thank you very much for your time and consideration. I eagerly look forward to your response.

[anicolson]

You should be able to use this in either your config or as a command line argument: https://github.com/csiro-mlai/dl_hpc_starter_pack/blob/352bf81e8d9aefcb31cfef2a9c4ce77504a394bc/src/dlhpcstarter/command_line_arguments.py#L79

--test-ckpt-path /path/to/last.ckpt

[anicolson]

It will get passed to:

cxrmate/tools/stages.py

Line 113 in 9aea762

args.exp_dir_trial, args.monitor, args.monitor_mode, args.test_epoch, args.test_ckpt_path,

And then...: https://github.com/csiro-mlai/dl_hpc_starter_pack/blob/352bf81e8d9aefcb31cfef2a9c4ce77504a394bc/src/dlhpcstarter/utils.py#L291

And it will eventually just be given as the ckpt path to...:

cxrmate/tools/stages.py

Line 119 in 9aea762

model = TaskModel.load_from_checkpoint(checkpoint_path=ckpt_path, **vars(args))

[yihp]

thanks very much for the detailed process, I will try it

[yihp]

hi @anicolson ,

I found that I had to delete the parameters --test-ckpt-name and --other-exp-dir in command_line_arguments.py and then pass in the parameters --test-ckpt-path to load the trained model last.ckpt for testing.
I think it is because of hasattr(args, 'test_ckpt_name') and hasattr(args, 'other_exp_dir'). As long as the parameter test_ckpt_name and other_exp_dir is included in command_line_arguments.py, Then the corresponding code logic will be executed without loading the trained model last.ckpt.
In addition, my dlhpcstarter is installed in \usr\local\lib\python3.8\site-packages\dlhpcstarter, which is not very convenient to modify. I am currently using the soft link method. Do you have a better way?

[anicolson]

What version of dlhpcstarter are you using? Can you try with 0.1.7?

[anicolson]

If it doesn't work, i'll have a look soon

[yihp]

The dlhpcstarter version I used was 0.1.7, but I delete the parameters --test-ckpt-name and --other-exp-dir in \usr\local\lib\python3.8\site-packages\command_line_arguments.py and it worked.

[anicolson]

Okay. I'll update the dlhpcstarter pack to fix that problem. Will let you know when its fixed.

[yihp]

That's great, thank u very much

[anicolson]

Hi, can you try 0.1.8? https://pypi.org/project/dlhpcstarter/

This passed, so hopefully it also works for you: https://github.com/csiro-mlai/dl_hpc_starter_pack/blob/efaae3e703492fabffcc69edfd8976f72d0fd1bd/tests/run_tests.sh#L5

[yihp]

thanks very much for the detailed process, I will try 0.1.8