Some questions about simplify

[SEU-NiuWenLong]

When I try the simplify process with the param.json template of the official website, I find that there are many parameters plus errors, such as cvasp,shuffle_poscar,train_param, etc., the following error types will occur, is it because of the problem with my dpgen version or the official website has not been updated? Secondly, when I used the following parameter files for simplify, my initial data was only 733, but in the second round of re-sampling, it was taken from more than 10,000 data. After I entered the 01.model_devi folder in the iter00001 folder, the details file recorded data results from 0 to 727, a total of 728, but cycled 24 times (the same data was repeated 24 times). Why? According to reason, it should be sampled from 728 results. Why is it repeated 24 times to get a lot of repetitive structures? I ran again, the second time is the same 728 data repeated 32 times, the number of repetitions in the two details files are not the same, why?
This is the param.json file I use.
{
"type_map": ["I","Pb","C","N","H"],
"mass_map": [126.90447, 207.2, 12.0108, 14.0067, 1.00795],

"pick_data": "../data",
"init_data_prefix": "",
"init_data_sys": [],
"sys_configs": [null],
"sys_batch_size": [1],

"_comment": " 00.train ",
"numb_models": 4,
"model_devi_activation_func":[["tanh","tanh"],["tanh","gelu"],["gelu","gelu"],["gelu","tanh"]],
"default_training_param": {
"model": {
"type_map": ["I","Pb","C","N","H"],
"descriptor" :{
"type": "se_e2_a",
"sel": [20, 8, 8, 8, 48],
"rcut_smth": 2,
"rcut": 6.0,
"neuron": [25, 50, 100],
"resnet_dt": false,
"axis_neuron": 12,
"seed": 1,
"_comment": " that's all"
},
"fitting_net" : {
"neuron": [240, 240, 240],
"resnet_dt": true,
"seed": 1,
"_comment": " that's all"
},
"_comment": " that's all"
},

      "learning_rate" :{
      "type":		"exp",
      "decay_steps":	25,
      "start_lr":	0.001,	
      "stop_lr":	3.51e-8,
      "_comment":	"that's all"
      },
  
      "loss" :{
      "_type":		"ener",
      "start_pref_e":	0.02,
      "limit_pref_e":	2,
      "start_pref_f":	1000,
      "limit_pref_f":	2,
      "start_pref_v":	0.01,
      "limit_pref_v":	1,
      "_comment":	" that's all"
      },
  "training": {
      "numb_steps":	5000,
      "disp_file": "lcurve.out",
      "disp_freq": 200,
      "numb_test": 4,
      "save_freq": 200,
      "save_ckpt": "model.ckpt",
      "disp_training": true,
      "time_training": true,
      "profiling": false,
      "profiling_file": "timeline.json",
      "_comment": "that's all"
  }

},
"_2comment": "02.fp",
"fp_style": "vasp",
"fp_skip_bad_box": "length_ratio:5;height_ratio:5",
"fp_accurate_threshold": 0.10,
"fp_accurate_soft_threshold": 0.00,
"fp_task_max": 20,
"fp_task_min": 5,
"ratio_failed": 0.30,
"fp_pp_path": "../vasp_inputs/",
"fp_pp_files": ["POTCAR_I","POTCAR_Pb_d","POTCAR_C","POTCAR_N","POTCAR_H"],
"fp_incar": "INCAR",

"labeled": false,
"init_pick_number":5,
"iter_pick_number":5,
"model_devi_f_trust_lo":0.30,
"model_devi_f_trust_hi":100.00,
"cvasp": false
}
This is my dpgen.log file.
2023-08-25 10:31:46,599 - INFO : start simplifying
2023-08-25 10:31:46,601 - INFO : continue from iter 000 task 08
2023-08-25 10:31:46,601 - INFO : =============================iter.000000==============================
2023-08-25 10:31:46,601 - INFO : =============================iter.000001==============================
2023-08-25 10:31:46,601 - INFO : -------------------------iter.000001 task 00--------------------------
2023-08-25 10:31:46,703 - INFO : -------------------------iter.000001 task 01--------------------------
2023-08-25 12:05:39,073 - INFO : -------------------------iter.000001 task 02--------------------------
2023-08-25 12:05:39,073 - INFO : -------------------------iter.000001 task 03--------------------------
2023-08-25 12:05:39,074 - INFO : -------------------------iter.000001 task 04--------------------------
2023-08-25 12:10:41,727 - INFO : -------------------------iter.000001 task 05--------------------------
2023-08-25 12:11:11,355 - INFO : candidate : 16368 in 17472 93.68 %
2023-08-25 12:11:11,355 - INFO : accurate : 1104 in 17472 6.32 %
2023-08-25 12:11:11,356 - INFO : failed : 0 in 17472 0.00 %
2023-08-25 12:11:11,358 - INFO : total candidate 16368 picked 5 ( 0.03 %) rest 16363 ( 99.97 % )
2023-08-25 12:11:35,260 - INFO : -------------------------iter.000001 task 06--------------------------
2023-08-25 12:11:35,271 - INFO : -------------------------iter.000001 task 07--------------------------
2023-08-25 15:49:03,881 - INFO : start simplifying
2023-08-25 15:49:03,889 - INFO : continue from iter 001 task 05
2023-08-25 15:49:03,889 - INFO : =============================iter.000000==============================
2023-08-25 15:49:03,889 - INFO : =============================iter.000001==============================
2023-08-25 15:49:03,889 - INFO : -------------------------iter.000001 task 06--------------------------
2023-08-25 15:49:03,993 - INFO : -------------------------iter.000001 task 07--------------------------
2023-08-25 17:20:26,349 - INFO : -------------------------iter.000001 task 08--------------------------
2023-08-25 17:20:26,583 - INFO : failed frame: 0 in 5 0.00 %
2023-08-25 17:20:26,603 - INFO : failed tasks: 0 in 5 0.00 %
2023-08-25 17:20:26,611 - INFO : =============================iter.000002==============================
This is an error caused by some parameters.

[SEU-NiuWenLong]

This is an error caused by some parameters: dargs.dargs.ArgumentKeyError: [at root location] undefined key 'cvasp'is not allowed in strict mode

[njzjz]

I don't reproduce this error using your parameter file.

[SEU-NiuWenLong]

This is part of the details file, each time step from 0 to 727, but repeated multiple times

data.rest.old/I12Pb4C4N4H24

step max_devi_v min_devi_v avg_devi_v max_devi_f min_devi_f avg_devi_f

       0       1.107616e-01       1.034090e-02       6.454280e-02       6.260143e-01       1.033384e-01       2.734852e-01
       1       8.406152e-02       8.497300e-03       4.604846e-02       4.961081e-01       7.248558e-02       1.836508e-01
       2       1.047533e-01       1.882767e-02       5.820302e-02       1.406245e+00       7.804563e-02       3.024227e-01
       3       3.191527e-02       7.599011e-03       2.195058e-02       6.432518e-01       4.327677e-02       1.753058e-01
       4       7.329263e-02       1.002962e-02       4.011213e-02       5.714682e-01       7.090985e-02       2.269617e-01
       5       3.351698e-02       3.497901e-03       2.052279e-02       3.511951e-01       4.872495e-02       1.299986e-01
       6       1.111205e-01       1.208831e-02       6.036468e-02       6.526824e-01       1.104913e-01       2.527558e-01
       7       4.274688e-02       4.353960e-03       2.499064e-02       3.867734e-01       6.658251e-02       1.749791e-01
       8       8.732767e-02       6.884538e-03       4.744665e-02       3.713601e-01       4.802477e-02       1.658989e-01
       9       1.022740e-01       1.026639e-02       4.695882e-02       3.202707e-01       5.650707e-02       1.499953e-01
      10       4.774948e-02       5.969635e-03       2.384989e-02       3.486393e-01       6.194128e-02       1.482693e-01
      11       3.087764e-02       1.271885e-03       1.534006e-02       2.939080e-01       4.973461e-02       1.215012e-01
      12       5.788511e-02       3.946744e-03       3.301411e-02       3.699964e-01       6.973095e-02       1.678886e-01
      13       3.524967e-02       4.034834e-03       2.387474e-02       4.546048e-01       8.709591e-02       1.817201e-01
      14       7.796376e-02       4.624730e-03       3.954954e-02       3.073889e-01       4.467168e-02       1.337180e-01

[njzjz]

You use MPI using 24 or 32 processors, making it printed 24 or 32 times. MPI should not be used. Besides, you should not use MPI when training on a single node.

[SEU-NiuWenLong]

Thank you for your answer. I saw this sentence on the official website(https://docs.deepmodeling.com/projects/deepmd/en/master/troubleshooting/howtoset_num_nodes.html): Parallelism for MPI is optional and used for multiple nodes, multiple GPU cards, or sometimes multiple CPU cores. The cluster I use has multiple CPU cores, so I tried the mpirun command to run dp and lmp. I tested that without mpirun, the training process of dp would be faster, but the process of lmp is very time-consuming. I have also tried to change environment variables, such as OMP_NUM_THREADS,TF_INTRA_OP_PARALLELISM_THREADS,TF_INTRA_OP_PARALLELISM_THREADS, etc., but it doesn't seem to have an accelerated effect, without mpirun. The dp command only accounts for about half of the CPU in the background process. Not making full use of resources.