Merge pull request #29 from BattModels/mkphuthi/develop

Mkphuthi/develop

.gitignore

@@ -157,3 +157,6 @@ cython_debug/
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
+
+# ASIMTools results
+examples/*/*results*/

Building_docs.md

@@ -0,0 +1,18 @@
+To build the documentation you will need the following packages:
+
+- sphinx
+- sphinx-apidoc
+- autodoc
+
+### Steps
+1. First, generate the autdoc sources for the API reference from the root using
+
+        sphinx-apidoc -f -o docs asimtools
+
+2. Then go into the docs directory and run
+
+        make html
+
+This will generate the docs in the `_build` directory
+
+

CONTRIBUTING.md

@@ -5,10 +5,10 @@ When contributing to this repository, please first discuss the change you wish t
 Please note we have a code of conduct, please follow it in all your interactions with the project.
 
 There are two main ways to contribute. One is to make changes to the existing
-codes and scripts. In this case, please make sure your change passes tests or
-submit a test accordinly. The second way to contribute is to add a new script
-to the core set of scripts. Reviewers will work with you to make sure your new
-script follows best practice including using utils, syntax highlighting, docs
+codes and asimmodules. In this case, please make sure your change passes tests or
+submit a test accordinly. The second way to contribute is to add a new asimmodule
+to the core set of asimmodules. Reviewers will work with you to make sure your new
+asimmodule follows best practice including using utils, syntax highlighting, docs
 and tests.
 
 ## Pull Request Process

README.md

@@ -1,166 +1,118 @@
+<!-- [![code coverage](https://img.shields.io/codecov/c/gh/materialsproject/jobflow/main)](https://codecov.io/gh/materialsproject/jobflow/) -->
+<!-- [![pypi version](https://img.shields.io/pypi/v/jobflow?color=blue)](https://pypi.org/project/jobflow/) -->
+<!-- ![supported python versions](https://img.shields.io/pypi/pyversions/jobflow) -->
+<!-- [![DOI](https://joss.theoj.org/papers/10.21105/joss.05995/status.svg)](https://doi.org/10.21105/joss.05995) -->
+
+[Documentation](https://battmodels.github.io/asimtools/) | [GitHub](https://github.com/BattModels/asimtools)
+
 # Atomic SIMulation Tools
 
 This package is a lightweight workflow and simulation manager for reproducible
 atomistic simulations that can be transferred across environments, calculators
-and structures. By using in-built or user-defined scripts and utilities, users
-can run/build their own simulation recipes and automagically scale them on
-slurm based clusters. The core idea is to separate the dependence of the
-atomistic potential/calculator and the simulations steps thereby allowing the
-same simulation to be run with multiple calculators and the same calculator to
-be used for multiple simulations. Input and output files must follow a strict
-format so that consistent analysis pipelines can be used across users
-
-** ASIMTools is going through some breaking changes for version 0.1.0, please use with caution **
-
-## Documentation
-Documentation is under construction [here](https://battmodels.github.io/asimtools/)
+and structures. By using in-built or user-defined asimmodules and utilities,
+users can run/build their own simulation recipes and automagically scale them
+locally or on slurm based clusters. The core idea is to separate the dependence
+of the atomistic potential/calculator and the simulations steps thereby
+allowing the same simulation to be run with multiple calculators and the same
+calculator to be used for multiple simulations without altering simulation
+code. Input and output files must follow a strict format so that consistent
+analysis pipelines can be used across users
 
 ## Developer philosophy
-The goal of asimtools is to push all the complexity of workflow
-management, best-practices, file management etc. into the backend such that the
-everyday user only has to handle input files for existing workflows and script
-files for workflows they want to implement. The following are the guiding
-principles for how asimtools should work:
-
-- Scripts should resemble boilerplate ASE code as much as possible.
-- Scripts should not explicitly depend on a calculator
-- Scripts should not explicitly depend on the context in which they run
-- It should be easy to debug individual scripts/parts in large workflows. In
-  addition, it should always be possible to debug/resubmit jobs without using
-  asimtools.
-- input file structure and format should be standard across all scripts. In
+The goal of asimtools is to push all the complexity of workflow management,
+best-practices, file management etc. into the backend such that the everyday
+user only has to handle input files for existing workflows and asimmodule files
+for workflows they want to implement. The following are the guiding principles
+for how asimtools should work:
+
+- Asimmodules should resemble boilerplate ASE code as much as possible.
+- Asimmodules should not explicitly depend on a calculator
+- Asimmodules should not explicitly depend on the context/environment in which 
+  they run
+- It should be easy to debug individual asimmodules/parts in large workflows.
+  In addition, it should always be possible to debug/resubmit jobs without
+  using asimtools.
+- Input file structure and format should be standard across all asimmodules. In
   addition all input parameters should match how they would like without
-  asimtools i.e. do not provide an API! This means extensive use of importlib.
-- output file structure should be predictable across all scripts
+  asimtools i.e. do not provide an API!
 - Job progress tracking must be incorporated
 - Best practices should be built-in e.g. if multiple jobs of the same slurm
-  context are submitted simulataneously, it must be a job array
+  context are submitted simulataneously, it must be a job array.
 
 ## Philosophy on User Experience
-The philosophy is to build "simulations" using building blocks of scripts.
-These scripts can be as complicated/efficient as you make them using any
-external packages you want and can be optimized with time but can still be run
-within the framework. This allows a test friendly way to transition from say a
-tutorial on the ASE/pymatgen website to an asimtools script. So while
-complicated wrappers are discouraged, they would still work as long as the
-script works. The benefit out of the box is that you can make your script
-independent of calculator or input structures and submit them easily.
+The philosophy is to build "simulations" using building blocks of asimmodules.
+Asimmodules are nothing but Python functions that return a dictionary, anything
+can be done in the function code. These asimmodules can be as
+complicated/efficient as you make them using any external packages you want and
+can be optimized with time but can still be run within the framework. This
+allows a test friendly way to transition from say a tutorial on the
+ASE/pymatgen website to an asimtools asimmodule. So while complicated wrappers
+are discouraged, they would still work as long as the asimmodule works. The
+benefit out of the box is that you can make your asimmodule independent of
+calculator or input structures and submit them easily.
 
 ## Getting Started
 
-These instructions will give you a copy of the project up and running on
-your local machine for development and testing purposes.
+These instructions will give you a copy of the project up and running.
 
 ### Installing
 You can install asimtools in a new conda environment using:
 ```
-conda create -n asimtools python=3.9
+conda create -n asimtools python=3.7
 conda activate asimtools
 
 conda install ase -c conda-forge
 
 git clone https://github.com/BattModels/asimtools.git
 cd asimtools
-pip install -e .
+pip install .
 ```
 
-This installs the base package in developer mode so that you do not have to
-reinstall if you make changes to the core package.
-
 Individual calculators may need external packages for loading those
-calculators. Similarly some scripts e.g. `lammps.py` might also need external packages to be used. It is up to the user to make sure those are installed.
+calculators. For example if you want to use Quantum Espresso or CASTEP, you
+will have to install them. Similarly some asimmodules e.g. `lammps.py` might
+also need external packages to be used. It is up to the user to make sure those
+are installed.
 
 You will also need to setup some environment variables, these variables point
-to a `env_input.yaml` and `calc_input.yaml` with your favorite configs since
-these are commonly shared among simulations. You can also directly specify them
-when running `asim-execute` (See `asim-execute -h`). 
-Examples of these files can be found in the examples.
+to global `env_input.yaml` and `calc_input.yaml` files with your favorite
+configurations since these are commonly shared among simulations. You can also
+directly specify them when running `asim-execute` (See `asim-execute -h`). You
+can also provide a directory for ASIMTools to search for your custom
+asimmodules. Examples of these files can be found in the examples.
 
 Add the following to your `.bashrc`
 ```
 export ASIMTOOLS_ENV_INPUT=/path/to/my/global/env_input.yaml
 export ASIMTOOLS_CALC_INPUT=/path/to/my/global/calc_input.yaml
-export ASIMTOOLS_SCRIPT_DIR=/path/to/my/script/dir
+export ASIMTOOLS_ASIMMODULE_DIR=/path/to/my/asimmodule/dir
 ```
-<!-- ## Examples -->
-<!-- Below are a few examples on how to run already implemented scripts. -->
-<!-- 
-The first thing to understand is the difference between `asim-execute sim_input.yaml` and 
-`asim-run sim_input.yaml`. The latter runs the chosen script in whatever location 
-and environment it happens to be launched from i.e. equivalent to 
-`python my_script.py sim_input.yaml` whereas the latter runs submits the job 
-that runs the script e.g. it will go to the work directory and launch a slurm job 
-from there containing `asim-run sim_input.yaml`.  `asim-execute` is essentially 
-"environment-aware" and runs the script in the environment specified by `env_id`
-whereas `asim-run` does not use `env_id` at all. -->
-
-<!-- A template is provided for writing a new script called `template.py`
-
-We provide a standard set of building block scripts for simulations/workflows that tend to form components of large simulations. Let us consider some examples. 
-To see details of their arguments (`args`), see their docstrings (which don't exist yet :( )
-
-1. The simplest are "unit" scripts which do not internally call other scripts 
-or depend on results from other scripts. 
-The simplest example is `singlepoint.py` which runs an energy/force/stress calculation
-on a single structure. To launch in your environment of choice specified by
-`env_id` in `sim_input.yaml` use `asim-execute sim_input.yaml`. See `asim-execute -h`. 
-For all scripts, you have the ability to specify the `env_input.yaml` and `calc_input.yaml` or 
-skip them to use the global files specified in `ASE_CALC_INPUT` and 
-`ASE_ENV_INPUT`. Note that many scripts don't need `calc_input.yaml` if they 
-just do some preprocessing or analysis. Another example is `atom_relax.py`.
-*Exercise: Add `cell_relax.py` based on `atom_relax.py`*
-
-2. Another type are "parent" scripts which launch multiple "child" scripts in parallel.
-An example is `image_array.py` which runs the same simulation on multiple
-images e.g. you want the energies of all the structures in a database. Note
-that you only need to point to the images, the env and the calc in the sim_input,
-The script, which uses a DistributedJob object automatically handles whether to
-submit jobs using a slurm array, individual slurm jobs or one after the other
-depending on the specified `env_id`. Another example is 
-`strong_scaling.strong_scaling.py`. 
-*Exercise: Implement `env_array.py` and `calc_array.py` based on `image_array.py` and `strong_scaling.py`.*
-
-3. The last major type of "parent" script are chained scripts. These scripts run one 
-**UnitJob** after the other based on a specified workflow. The job automatically runs
-one job after another or submits slurm jobs with appropriate dependencies. Note
-that if one of the unitjobs internally calls another script such as 
-`image_array.py`, then the slurm dependencies will fail but everything else should work.
-In this casea, the current workaround is to set `submit=false` for the stages of 
-the chain that would fail due to a previous array/chain job. Then rerun `asim-execute`
-again with `submit=true`, It should automatically skip the completed steps. See `eos.eos.py` 
-and the corresponding example. An example to help understand the genera; chained sim_input is 
-`chained`
-
-4. Hybrid jobs combine multiple of these elements e.g. `eos.eos.py` runs a
-`preprocessing` script to prepare the scaled structures followed by a UnitJob that submits
-an `image_array` for each scaled structure and finally a unitjob to `postprocess`
-the results from the array. It is possible to 
-- build hybrid scripts in python without using existing asimtools scripts
-- running a `chained.py` on a `sim_input.yaml` with the correct format that runs scripts that are already defined
-- to write a script such that it directly manipulates Job objects. This is the most flexible and robust but ofcourse most complicated
-*Exercise: Run the eos calculation without writing any new scripts, simply using the scripts in the core (image_array) and eos subfolder (preprocessing and postprocessing).* 
-The key is being able to to point to the correct files for each step before the calculation is run. -->
 
 ## Running the tests
 
-To run all tests from the tests directory, call:
+To run tests for the workflow tools, from the tests directory, call:
 
     pytest
 
 To run the test suite on a component `component.py` , call:
 
     pytest test_component.py
-
-<!-- ## Basic example
 
-Simulations are run by providing a `*calc_input.yaml` and `*sim_input.yaml` file which specify 
-the calculator (and the environment it runs in) and the simulation parameters which are specific 
-to the simulation being run. The recommended method for calling scripts is to use
+To run all tests for the provided asimmodules, `cd` into the examples directory
+and call:
 
-```
-asim-run *calc_input.yaml *sim_input.yaml
-``` -->
+    source run_all.sh
+
+Or you can run a test for each individual example in its directory using:
+
+    source run.sh
 
+If no errors are reported, the tests have passed. These tests simply test the
+functionality of the code, not the validity of the simulations!
+
+To run tests for the provided asimmodules using slurm, you can similarly run
+the bash scripts ending with `_slurm.sh`.
+
 ## Contributing
 
 Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code

asimtools/scripts/parity.py → asimtools/asimmodules/benchmarking/parity.py

@@ -2,7 +2,7 @@
 '''
 Generates a parity plot and collects evaluation statistics comparing energy
 and/or forces and/or stress to existing values in the provided dataset. This
-script can work in parallel based on the number of cores specified.
+asimmodule can work in parallel based on the number of cores specified.
 
 Author: [email protected]
 

asimtools/scripts/do_nothing.py → asimtools/asimmodules/do_nothing.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 '''
-Script for unit tests and debugging, does nothing for specified duration
+asimmodule for unit tests and debugging, does nothing for specified duration
 '''
 from time import sleep
 from typing import Dict, Optional

asimtools/scripts/elastic_constants/cubic_energy_expansion.py → asimtools/asimmodules/elastic_constants/cubic_energy_expansion.py

@@ -1,9 +1,9 @@
 #!/usr/bin/env python
 '''
-Describe the script briefly here. If this is a script that runs multiple steps,
+Describe the asimmodule briefly here. If this is a asimmodule that runs multiple steps,
 describe it here using reStructuredText to generate autodocs
 
-Cite the papers where the method/script was first introduced here as well
+Cite the papers where the method/asimmodule was first introduced here as well
 
 Author: [email protected]
 '''
@@ -17,10 +17,9 @@
 from asimtools.utils import (
     get_atoms,
 )
-from asimtools.scripts.ase_cubic_eos_optimization import (
+from asimtools.asimmodules.geometry_optimization.ase_cubic_eos_optimization import (
     ase_cubic_eos_optimization as eos
 )
-from asimtools.scripts.cell_relax import cell_relax
 
 def get_strained_atoms(atoms, strain: str, delta: float):
     """Returns a unit cell with atoms strained according to some useful types
@@ -84,7 +83,7 @@ def cubic_energy_expansion(
     calc_id: str,
     image: Dict,
     deltas: Sequence[float] = (-0.01,-0.0075,-0.005,0.00,0.005,0.0075,0.01),
-    ase_eos_args: Optional[Dict] = None,
+    ase_cubic_eos_args: Optional[Dict] = None,
 ) -> Dict:
     '''
     Calculates B, C11, C12 and C44 elastic constants of a structure with cubic
@@ -98,8 +97,8 @@ def cubic_energy_expansion(
     # Start by getting the Bulk modulus and optimized cell from the EOS
     logging.info('Calculating EOS')
     eos_kwargs = {'image': image, 'calc_id': calc_id}
-    if ase_eos_args is not None:
-        eos_kwargs.update(ase_eos_args)
+    if ase_cubic_eos_args is not None:
+        eos_kwargs.update(ase_cubic_eos_args)
     eos_results = eos(**eos_kwargs)
     B = eos_results['B']
     atoms = get_atoms(image_file='eos_image_output.xyz')
@@ -135,7 +134,7 @@ def cubic_energy_expansion(
         c11min12_atoms.write(f's{delta:.4f}_c11min12.xyz')
 
     def f(x, a, b, c):
-        ''' Fitting function for Free energy expansion to second order'''
+        ''' Fitting function for free energy expansion to second order'''
         return a*x**2 + b*x + c
 
     logging.info('Fitting for C44')
@@ -152,13 +151,15 @@ def f(x, a, b, c):
     anis = 2 * C44 / (C11 - C12)
 
     results = {
-        'B': float(B / GPa),
-        'C11minC12': float(C11minC12 / GPa),
-        'C11': float(C11 / GPa),
-        'C12': float(C12 / GPa),
-        'C44': float(C44 / GPa),
-        'Anisotropy': float(anis),
-        'vol': float(atoms.get_volume()),
+        'constants': {
+            'B': float(B / GPa),
+            'C11minC12': float(C11minC12 / GPa),
+            'C11': float(C11 / GPa),
+            'C12': float(C12 / GPa),
+            'C44': float(C44 / GPa),
+            'Anisotropy': float(anis),
+            'vol': float(atoms.get_volume()),
+        },
     }
 
     return results # Always return a dictionary! Use {} if necessary