diff --git a/reann/example/co2+ni100/job-mpi.sh b/example/co2+ni100/job-mpi.sh similarity index 100% rename from reann/example/co2+ni100/job-mpi.sh rename to example/co2+ni100/job-mpi.sh diff --git a/reann/example/co2+ni100/nn.err b/example/co2+ni100/nn.err similarity index 100% rename from reann/example/co2+ni100/nn.err rename to example/co2+ni100/nn.err diff --git a/reann/example/co2+ni100/para/input_density b/example/co2+ni100/para/input_density similarity index 100% rename from reann/example/co2+ni100/para/input_density rename to example/co2+ni100/para/input_density diff --git a/reann/example/co2+ni100/para/input_nn b/example/co2+ni100/para/input_nn similarity index 97% rename from reann/example/co2+ni100/para/input_nn rename to example/co2+ni100/para/input_nn index 08dc641..717d57c 100644 --- a/reann/example/co2+ni100/para/input_nn +++ b/example/co2+ni100/para/input_nn @@ -39,4 +39,4 @@ oc_table_norm=True oc_loop=3 #========================floder used to save the data======================== - folder="/home/home/zyl/pytorch/2021_9_5/data/co2+ni100/" + folder="../data/co2+ni100/" diff --git a/picture/data.jpg b/picture/data.jpg new file mode 100644 index 0000000..d2bbf61 Binary files /dev/null and b/picture/data.jpg differ diff --git a/picture/workflow.jpg b/picture/workflow.jpg new file mode 100644 index 0000000..fe571fc Binary files /dev/null and b/picture/workflow.jpg differ diff --git a/readme.md b/readme.md index 16ff15e..76c7e4f 100644 --- a/readme.md +++ b/readme.md @@ -1,24 +1,55 @@ -Recursively embedded atom neural network -================================================= -**Introduction:** -___________________________ - Recursively embedded atom neural network (REANN) is a PyTorch-based end-to-end multi-functional Deep Neural Network Package for Molecular, Reactive and Periodic Systems. Currently, REANN can be used to train interatomic potentials, dipole moments, transition dipole moments, and polarizabilities. Taking advantage of Distributed DataParallel features embedded in PyTorch, the training process is highly parallelized on both GPU and CPU. For the convenience of MD simulation, an interface to LAMMPS has been constructed by creating a new pair_style invoking this representation for highly efficient MD simulations. More details can be found in the manual saved in the "manual" folder and an example has been placed in "reann/example/" for constructing the ML potential of the co2+Ni(100) system using the data located in the "data/" folder. - -As a calculator, REANN can be used with Atomic Simulation Environment (ASE), you can read the readme in ASE/ folder for interface and the website to know how to use [ASE](https://wiki.fysik.dtu.dk/ase/) +#Recursively embedded atom neural network +##Introduction +Recursively embedded atom neural network (REANN) is a PyTorch-based end-to-end multi-functional Deep Neural Network Package for Molecular, Reactive and Periodic Systems. Currently, REANN can be used to train interatomic potentials, dipole moments, transition dipole moments, and polarizabilities. Taking advantage of Distributed DataParallel features embedded in PyTorch, the training process is highly parallelized on both GPU and CPU. For the convenience of MD simulation, an interface to LAMMPS has been constructed by creating a new pair_style invoking this representation for highly efficient MD simulations. In addition, REANN have been interfaced to ASE(https://wiki.fysik.dtu.dk/ase/) package as a calculator. More detials can be found on the manua -**Requirements:** -___________________________________ -* PyTorch 2.0.0 -* LibTorch 2.0.0 -* cmake 3.1.0 -* opt_einsum 3.2.0 +Field-induced REANN ([FIREANN](https://github.com/zhangylch/FIREANN.git)) developed based on the REANN package and can describes the response of the potential energy to an external field up to an arbitrary order (dipole moments, polarizabilities …) in a unified framework. + +##Requirements +1. PyTorch 2.0.0 +2. LibTorch 2.0.0 +3. cmake 3.1.0 +4. opt_einsum 3.2.0 -**Data sample:** -___________________________________ +##Data sample The REANN package has been embedded in [GDPy](https://github.com/hsulab/GDPy), which is used to search the configuration space and sample suitable configurations to construct machine learning potential functions. -**References:** -__________________________________________________ +## Training Workflow +The training process can be divided into four parts: information loading, initialization, dataloader and optimization. First, the "src.read" will load the information about the systems and NN structures from the dataset and input files (“input_nn” and “input_density”) respectivrly. Second, the "run.train" module utilizes the loaded information to initialize various classes, including property calculator, dataloader, and optimizer. For each process, an additional thread will be activated in the "src.dataloader" module to prefetch data from CPU to GPU in an asynchronous manner. Meanwhile, the optimization will be activated in the "src.optimize" module once the first set of data is transferred to the GPU. During optimization, a learning rate scheduler, namely "ReduceLROnPlateau" provided by PyTorch, is used to decay the learning rate. Training is stopped when the learning rate drops below "end_lr" and the model that performs best on the validation set is saved for further investigation. ![image](https://github.com/zhangylch/REANN/blob/main/picture/workflow.jpg) + +## How to Use REANN Package +Users can employ geometries, energies, atomic force vectors (or some other physical properties which are invariant under rigid translation, rotation, and permutation of identical atoms and their corresponding gradients) to construct a model. There are three routines to use this package: +1. [Prepare the environment](#Prepare-the-environment) +2. [Prepare data](#Prepare-data) +3. [Set up parameters](#Set-up-parameters) + +### Prepare the environment +The REANN Package is built based on PyTorch and uses the "opt_einsum" package for optimizing einsum-like expressions frequently used in the calculation of the embedded density. In order to run the REANN package, users need to install PyTorch (version: 2.0.0) based on the instructions on the [PyTorch](https://pytorch.org/get-started/locally/) official site and the package named [opt_einsum](https://optimized-einsum.readthedocs.io/en/stable/). + +### Prepare data +There are two directories that users need to prepare, namely, “train” and “val”, each of which includes a file “configuration” used to preserve the required information including lattice parameters, periodic boundary conditions, configurations, energy and atomic forces (if needed), dipole moments, polarizabilities, etc. For example, users want to represent the NMA system that has available atomic forces. The file "configuration" should be written in the following format.![image](https://github.com/zhangylch/REANN/blob/main/picture/data.jpg) +The first line can be an arbitrary character other than a blank line. The next three lines are the lattice vectors defining the unit cell of the system. The fifth line is used to enable(1)/disable(0) the periodic boundary conditions in each direction. In this example, NMA is not a periodic system, the fifth line should be “pbc 0 0 0”. For some gas-surface systems, only the x-y plane is periodic and the corresponding fifth line is “pbc 1 1 0”. Following N lines (N is the number of atoms in the system, here is 12): the columns from the left to right represent the atomic name, relative atomic mass, coordinates(x, y, z) of the geometry, atomic force vectors (if the force vector is not incorporated in the training, these three columns can be omitted). Next line: Start with "abprop:" and then follow bytarget property (energy/dipole/polzrizability). One example is stored in "data" folder. + +### Set up parameters +In the section, we will introduce some hyparameters concerning the embedded density and NN structures that are essential for obtianing an exact representation. More detailed introduction of all parameters can be found on the manual in the "manual" floder. These parameters are set up in two files "input_nn" and "input_density" saved in the "para" floder of your work directory. one example of input_nn and input_density is placed in the "example" foleder. +#### input_nn +1. batchsize_train=64 # required parameters type: integer +2. batchsize_val=128 # required parameters type: integer +(Number of configurations used in each batch for train (batchsize_train) and validation (batchsize_val). Note, this "batchsize_train" is a key parameter concerned with efficiency. Normally, a large enough value is given to achieve high usage of the GPU and lead to higher efficiency in training if you have sufficient data. However, for small training data, a large "batchsize" can lead to a decrease in accuracy, probably owing to the decrease in the number of gradient descents during each epoch. The decrease in accuracy may be compensated by more epochs (increase the "patience_epoch" ) or a larger learning rate. Some detailed testing is required here to achieve a balance of accuracy and efficiency in training. The value of "batch_val" has no effect on accuracy, and thus a larger value is preferred.) +3. oc_loop = 1 # type: integer +(Number of iterations used to represent the orbital efficients.) + +#### input_density +1. cutoff = 4.5 # type: real number +(Cutoff distances) +2. nipsin= 2 # type: integer +(Maximal angular momenta determine the orbital type (s, p, d ..)) +3. nwave=8 # type: integer +(Number of radial Gaussian functions. This number should be a power of 2 for better efficiency.) + +## MD simulations +As mentioned earlier, the package interfaces with the LAMMPS framework by creating a new pair_style (fireann).MD simulations can be run in a multi-process or multi-threaded fashion on both GPUs and CPUs. MD simulations based on other MD packages such as i-pi can also be executed through the existing ipi-lammps interface. In addition, MD simulation can also be performed by the ASE interface. More details can be found in the manual. + +##References If you use this package, please cite these works. 1. The original EANN model: Yaolong Zhang, Ce Hu and Bin Jiang *J. Phys. Chem. Lett.* 10, 4962-4967 (2019). 2. The EANN model for dipole/transition dipole/polarizability: Yaolong Zhang Sheng Ye, Jinxiao Zhang, Jun Jiang and Bin Jiang *J. Phys. Chem. B* 124, 7284–7290 (2020). diff --git a/reann/__pycache__/__main__.cpython-310.pyc b/reann/__pycache__/__main__.cpython-310.pyc index 68e5f18..28fdae6 100644 Binary files a/reann/__pycache__/__main__.cpython-310.pyc and b/reann/__pycache__/__main__.cpython-310.pyc differ diff --git a/reann/example/NMA/job-mpi.sh b/reann/example/NMA/job-mpi.sh deleted file mode 100644 index 910d044..0000000 --- a/reann/example/NMA/job-mpi.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/sh -#PBS -V -#PBS -q gpu -#PBS -N NMA-polar -#PBS -l nodes=1:ppn=1 -source /share/home/bjiangch/group-zyl/.bash_profile -# conda environment -conda_env=PyTorch-190 -export OMP_NUM_THREADS=16 -#path to save the code -path="/home/home/zyl/pytorch/2021_8_1/eann-8/" - -#Number of processes per node to launch -NPROC_PER_NODE=1 - -#Number of process in all modes -WORLD_SIZE=`expr $PBS_NUM_NODES \* $NPROC_PER_NODE` - -MASTER=`/bin/hostname -s` - -MPORT=`ss -tan | awk '{print $5}' | cut -d':' -f2 | \ - grep "[2-9][0-9]\{3,3\}" | sort | uniq | shuf -n 1` - -#You will want to replace this -COMMAND="$path " -conda activate $conda_env -cd $PBS_O_WORKDIR -#python3 -m torch.distributed.run --nproc_per_node=$NPROC_PER_NODE --max_restarts=0 --nnodes=$PBS_NUM_NODES --rdzv_id=$PBS_JOBID --rdzv_backend=c10d --rdzv_endpoint=$MASTER:$MPORT $COMMAND > out -python3 -m torch.distributed.run --nproc_per_node=$NPROC_PER_NODE --max_restarts=0 --nnodes=1 --standalone $COMMAND > out - diff --git a/reann/example/NMA/para/input_density b/reann/example/NMA/para/input_density deleted file mode 100644 index 43cd61f..0000000 --- a/reann/example/NMA/para/input_density +++ /dev/null @@ -1,6 +0,0 @@ -#==============param for atomic energy======================= - neigh_atoms=12 - cutoff=5e0 - nipsin=2 - atomtype=["C","H","N","O"] - nwave=7 diff --git a/reann/example/NMA/para/input_nn b/reann/example/NMA/para/input_nn deleted file mode 100644 index 10ad237..0000000 --- a/reann/example/NMA/para/input_nn +++ /dev/null @@ -1,42 +0,0 @@ -# required parameters - start_table = 1 # start_table table for the fit with force(1) or without force(0) - # start_table table for DM(2), TDM(3), polarizability(4) - table_coor = 0 # table_coor 0 for cartesian coordinates and 1 for direct coordinates - nl = [32,32] # neural network architecture - # nl = [512,512,512,512] # neural network architecture - nblock = 1 - dropout_p=[0.0,0.0] - table_init = 1 # 1 used for load parameters from pth - nkpoint=1 # number of nkpoint NNs was employed to representation polarizability -# NN epoch and NN optimize parameters - Epoch=20000 # max iterations epoch - patience_epoch = 200 # pre initial learning rate epoch - decay_factor = 0.5 # Factor by which the learning rate will be reduced. new_lr = lr * factor. - start_lr = 0.001 # initial learning rate - end_lr =1e-5 # final learning rate - re_ceff = 0 # factor for regularization -# wave epoch and wave optimize parameters - ratio = 0.9 # ratio for training - Find_unused=True -# ===================================================================== - batchsize = 128 # batch_size - e_ceff=0.1 - init_f=50 # init_f - final_f=0.5 # final_f -#========================queue_size sequence for laod data into gpu - queue_size=10 - print_epoch=5 - table_norm=True - DDP_backend='nccl' - activate = 'Relu_like' - dtype="float32" -#===========param for orbital coefficient =============================================== - oc_nl = [32,32] # neural network architecture - oc_nblock = 1 - oc_dropout_p=[0,0,0,0] - oc_activate = 'Relu_like' - #========================queue_size sequence for laod data into gpu - oc_table_norm=True - oc_loop=4 -#========================floder used to save the data======================== - floder="./" # data can be found in repo "https://github.com/zhangylch/Learning-Dipole-Moments-and-Polarizabilities" diff --git a/reann/example/NMA/readme b/reann/example/NMA/readme deleted file mode 100644 index 855ffae..0000000 --- a/reann/example/NMA/readme +++ /dev/null @@ -1 +0,0 @@ -Here is an example for fitting the polar of NMA and the corresponding dataset can be found in the repo "https://github.com/zhangylch/Learning-Dipole-Moments-and-Polarizabilities". diff --git a/reann/inference/__pycache__/density.cpython-310.pyc b/reann/inference/__pycache__/density.cpython-310.pyc index 4893007..f98a9f3 100644 Binary files a/reann/inference/__pycache__/density.cpython-310.pyc and b/reann/inference/__pycache__/density.cpython-310.pyc differ diff --git a/reann/inference/__pycache__/get_neigh.cpython-310.pyc b/reann/inference/__pycache__/get_neigh.cpython-310.pyc index 52d971b..25137a1 100644 Binary files a/reann/inference/__pycache__/get_neigh.cpython-310.pyc and b/reann/inference/__pycache__/get_neigh.cpython-310.pyc differ diff --git a/reann/lammps_REANN/__pycache__/PES.cpython-310.pyc b/reann/lammps_REANN/__pycache__/PES.cpython-310.pyc index 877cfc3..db4e325 100644 Binary files a/reann/lammps_REANN/__pycache__/PES.cpython-310.pyc and b/reann/lammps_REANN/__pycache__/PES.cpython-310.pyc differ diff --git a/reann/lammps_REANN/__pycache__/density.cpython-310.pyc b/reann/lammps_REANN/__pycache__/density.cpython-310.pyc index 7a47117..328e312 100644 Binary files a/reann/lammps_REANN/__pycache__/density.cpython-310.pyc and b/reann/lammps_REANN/__pycache__/density.cpython-310.pyc differ diff --git a/reann/lammps_REANN/__pycache__/script_PES.cpython-310.pyc b/reann/lammps_REANN/__pycache__/script_PES.cpython-310.pyc index 0dc8696..fdd62d1 100644 Binary files a/reann/lammps_REANN/__pycache__/script_PES.cpython-310.pyc and b/reann/lammps_REANN/__pycache__/script_PES.cpython-310.pyc differ diff --git a/reann/pes/__pycache__/PES.cpython-310.pyc b/reann/pes/__pycache__/PES.cpython-310.pyc index 64ac062..7d165da 100644 Binary files a/reann/pes/__pycache__/PES.cpython-310.pyc and b/reann/pes/__pycache__/PES.cpython-310.pyc differ diff --git a/reann/pes/__pycache__/script_PES.cpython-310.pyc b/reann/pes/__pycache__/script_PES.cpython-310.pyc index 9090b6a..8f4a171 100644 Binary files a/reann/pes/__pycache__/script_PES.cpython-310.pyc and b/reann/pes/__pycache__/script_PES.cpython-310.pyc differ diff --git a/reann/run/__pycache__/train.cpython-310.pyc b/reann/run/__pycache__/train.cpython-310.pyc index 64048e0..ed9267e 100644 Binary files a/reann/run/__pycache__/train.cpython-310.pyc and b/reann/run/__pycache__/train.cpython-310.pyc differ diff --git a/reann/src/__pycache__/EMA.cpython-310.pyc b/reann/src/__pycache__/EMA.cpython-310.pyc index 2b437a8..32afbb7 100644 Binary files a/reann/src/__pycache__/EMA.cpython-310.pyc and b/reann/src/__pycache__/EMA.cpython-310.pyc differ diff --git a/reann/src/__pycache__/Loss.cpython-310.pyc b/reann/src/__pycache__/Loss.cpython-310.pyc index b21ff4f..603df8c 100644 Binary files a/reann/src/__pycache__/Loss.cpython-310.pyc and b/reann/src/__pycache__/Loss.cpython-310.pyc differ diff --git a/reann/src/__pycache__/MODEL.cpython-310.pyc b/reann/src/__pycache__/MODEL.cpython-310.pyc index 894566c..73682f7 100644 Binary files a/reann/src/__pycache__/MODEL.cpython-310.pyc and b/reann/src/__pycache__/MODEL.cpython-310.pyc differ diff --git a/reann/src/__pycache__/Property_energy.cpython-310.pyc b/reann/src/__pycache__/Property_energy.cpython-310.pyc new file mode 100644 index 0000000..219f20d Binary files /dev/null and b/reann/src/__pycache__/Property_energy.cpython-310.pyc differ diff --git a/reann/src/__pycache__/activate.cpython-310.pyc b/reann/src/__pycache__/activate.cpython-310.pyc index 9bffa7e..307db83 100644 Binary files a/reann/src/__pycache__/activate.cpython-310.pyc and b/reann/src/__pycache__/activate.cpython-310.pyc differ diff --git a/reann/src/__pycache__/com.cpython-310.pyc b/reann/src/__pycache__/com.cpython-310.pyc index c21a188..bee3da5 100644 Binary files a/reann/src/__pycache__/com.cpython-310.pyc and b/reann/src/__pycache__/com.cpython-310.pyc differ diff --git a/reann/src/__pycache__/cpu_gpu.cpython-310.pyc b/reann/src/__pycache__/cpu_gpu.cpython-310.pyc index 8a42297..25d1263 100644 Binary files a/reann/src/__pycache__/cpu_gpu.cpython-310.pyc and b/reann/src/__pycache__/cpu_gpu.cpython-310.pyc differ diff --git a/reann/src/__pycache__/dataloader.cpython-310.pyc b/reann/src/__pycache__/dataloader.cpython-310.pyc index 8aeb9df..185e108 100644 Binary files a/reann/src/__pycache__/dataloader.cpython-310.pyc and b/reann/src/__pycache__/dataloader.cpython-310.pyc differ diff --git a/reann/src/__pycache__/density.cpython-310.pyc b/reann/src/__pycache__/density.cpython-310.pyc index 80bf77d..a1abc9a 100644 Binary files a/reann/src/__pycache__/density.cpython-310.pyc and b/reann/src/__pycache__/density.cpython-310.pyc differ diff --git a/reann/src/__pycache__/get_batchneigh.cpython-310.pyc b/reann/src/__pycache__/get_batchneigh.cpython-310.pyc index b6acfb5..90894b5 100644 Binary files a/reann/src/__pycache__/get_batchneigh.cpython-310.pyc and b/reann/src/__pycache__/get_batchneigh.cpython-310.pyc differ diff --git a/reann/src/__pycache__/get_info_of_rank.cpython-310.pyc b/reann/src/__pycache__/get_info_of_rank.cpython-310.pyc index edf4d16..91bcebc 100644 Binary files a/reann/src/__pycache__/get_info_of_rank.cpython-310.pyc and b/reann/src/__pycache__/get_info_of_rank.cpython-310.pyc differ diff --git a/reann/src/__pycache__/get_neighbour.cpython-310.pyc b/reann/src/__pycache__/get_neighbour.cpython-310.pyc index 932b8d8..8daf092 100644 Binary files a/reann/src/__pycache__/get_neighbour.cpython-310.pyc and b/reann/src/__pycache__/get_neighbour.cpython-310.pyc differ diff --git a/reann/src/__pycache__/gpu_sel.cpython-310.pyc b/reann/src/__pycache__/gpu_sel.cpython-310.pyc index 739650b..e0c831d 100644 Binary files a/reann/src/__pycache__/gpu_sel.cpython-310.pyc and b/reann/src/__pycache__/gpu_sel.cpython-310.pyc differ diff --git a/reann/src/__pycache__/optimize.cpython-310.pyc b/reann/src/__pycache__/optimize.cpython-310.pyc index 37d9229..94b3550 100644 Binary files a/reann/src/__pycache__/optimize.cpython-310.pyc and b/reann/src/__pycache__/optimize.cpython-310.pyc differ diff --git a/reann/src/__pycache__/read.cpython-310.pyc b/reann/src/__pycache__/read.cpython-310.pyc index 9ff2404..48c18a7 100644 Binary files a/reann/src/__pycache__/read.cpython-310.pyc and b/reann/src/__pycache__/read.cpython-310.pyc differ diff --git a/reann/src/__pycache__/read_data.cpython-310.pyc b/reann/src/__pycache__/read_data.cpython-310.pyc index e3e1c9a..1b8253f 100644 Binary files a/reann/src/__pycache__/read_data.cpython-310.pyc and b/reann/src/__pycache__/read_data.cpython-310.pyc differ diff --git a/reann/src/__pycache__/restart.cpython-310.pyc b/reann/src/__pycache__/restart.cpython-310.pyc index 18ad3da..369f88d 100644 Binary files a/reann/src/__pycache__/restart.cpython-310.pyc and b/reann/src/__pycache__/restart.cpython-310.pyc differ diff --git a/release_note b/release_note index 8878e58..5e9ff6f 100644 --- a/release_note +++ b/release_note @@ -1 +1 @@ -2022_2_17 commit: Fixed a bug that could occur when an unequal number of batches of different processes occurred during multi-process training. +v2.0 commit: Improve the efficiency of reann's interface with ASE and gain similar efficiency to the lammps interface.