From 0808eec3786f44715ec8b092439f938500974019 Mon Sep 17 00:00:00 2001 From: "Ryan M. Richard" Date: Mon, 30 Sep 2024 22:36:15 -0500 Subject: [PATCH 1/4] backup [skip ci] --- docs/_data/navigation.yml | 2 + docs/_pages/about.md | 68 +++++++++++++++++++++++++++++++ docs/_pages/developer/home.md | 13 +++++- docs/_pages/developer/overview.md | 35 ++++++++++++++++ docs/_pages/home.md | 10 ++++- 5 files changed, 125 insertions(+), 3 deletions(-) create mode 100644 docs/_pages/about.md create mode 100644 docs/_pages/developer/overview.md diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index cf93a42..ed66b7b 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -1,4 +1,6 @@ main: + - title: "About" + url: "/about/" - title: "Plugins" url: "/plugins/" - title: "Community Members" diff --git a/docs/_pages/about.md b/docs/_pages/about.md new file mode 100644 index 0000000..fac679d --- /dev/null +++ b/docs/_pages/about.md @@ -0,0 +1,68 @@ +--- +title: "About the NWChemEx Project" +layout: splash +permalink: /about/ +--- + +The short of it is NWChemEx is a quantum chemistry software package which +strives to adhere to modern software engineering best practices. These practices +champion a separation-of-concerns approach which results in a package with many +well-defined pieces. Users benefit from the "app-store" like experience +and developers benefit by being able to quickly inject code throughout the +software for rapid prototyping, custom workflows, and performance tuning. + +## NWChemEx Focuses on Quantum Chemistry +Chemistry is the study of matter, i.e., things made of atoms. The properties of +matter stem from how the constituent atoms interact. Given an arrangement of +atoms, computational chemistry strives to accurately compute the interactions +among the atoms and therefore the resulting properties. While the +equations for computing the atomic interactions for a set of atoms are known, +even with today's supercomputers these equations can only be exactly solved +for systems containing a few atoms. Most systems of interest in chemistry +contain many molecules interacting with each other and usually each of the +molecules contains somewhere on the order of 10 to 100 atoms. Exact solution +for systems of chemical interest is unlikely to be possible any time soon. + +NWChemEx is targeted at providing high-performance implementations of quantum +chemistry methods. Generally speaking, quantum chemistry methods remain +accurate by staying true to the physics of the system, but are also practical +because they only treat a subset of interactions. While more practical than +the exact equations, most quantum chemistry methods still require a substantial +amount of computational resources. NWChemEx actively strives to increase the +applicability of quantum chemistry methods by developing new reduced cost +algorithms and by taking full advantage of today's modern hardware. + +## NWChemEx Philosophy + +### User Friendly + +At launch NWChemEx has a limited selection of features; however, we strive to +make using those features easy for users regardless of whether the user's use +case is a heroic exascale computation, a high-throughput campaign, or a +teaching exercise for undergraduate physical chemistry. + +For more details see [NWChemEx Community](/community/). + +### Ecosystem Friendly +Many software developers within scientific computing are embracing modularity. +This means there is an abundance of existing, open-source, performant software +which can be leveraged in an algorithm instead of needing to "reinvent the +wheel." To capitalize on this, NWChemEx is built on a plugin system which makes +it easy for users to inject code ---be it an existing library, custom behavior, +or an entire prototype routine --- into existing algorithms. Moreover this can +be accomplished without needing to directly modify the existing code base. + +For more details see [NWChemEx Developers](/developer/). + +### Theory Friendly +At present the majority of NWChemEx's development has been on infrastructure +with the goal of making developing electronic structure theory easier. To that +end, NWChemEx embraces object-oriented programming (OOP). Using OOP it is much +easier to decouple the developer's intent from the optimization of that intent. +For example, using objects meant to represent wavefunctions and operators, the +developer can write a quantum chemistry method in a manner similar to what one +finds in a textbook. Under the hood, the wavefunction and operator classes can +dispatch to routines which know how to efficiently form, compose, and operate +on tensors. + +For more details see [NWChemEx Authors](/author/). \ No newline at end of file diff --git a/docs/_pages/developer/home.md b/docs/_pages/developer/home.md index 1e89590..b1a0078 100644 --- a/docs/_pages/developer/home.md +++ b/docs/_pages/developer/home.md @@ -4,4 +4,15 @@ layout: splash permalink: /developer/ --- -TODO: content \ No newline at end of file +The NWChemEx community views anyone who writes software which interfaces with +NWChemEx as an NWChemEx developer. That said, the topics in this section will be +most useful if you are: + +- A developer/maintainer of an existing library, tool, or software package and + you want to make the library, tool, or software package interoperable with + the NWChemEx ecosystem. +- A software developer looking to create a library, tool, or software package + and you want the library, tool, or software package to be readily usable by + the both the NWChemEx ecosystem and the broader community. + + \ No newline at end of file diff --git a/docs/_pages/developer/overview.md b/docs/_pages/developer/overview.md new file mode 100644 index 0000000..db06275 --- /dev/null +++ b/docs/_pages/developer/overview.md @@ -0,0 +1,35 @@ +--- +title: "Overview of NWChemEx Development" +layout: page +permalink: /developer/overview/ +--- + +The NWChemEx ecosystem is predicated on the assumption that developers are +interested in creating and consuming modular software. We begin with an +overview of what modular software is. From there we sketch out the high-level +steps needed to create a plugin or extend an existing plugin. The contents of +this page are conceptual links to concrete tutorials are provided throughout. + +## What is a modular software package? + +- Idea is to explain what a modular software package looks like and to + introduce the basic considerations of designing such a package. +- Key point: achieved by worrying about composing quantities, and not how the + quantity is computed. + +## What is a software module? + +- Introduce as opposite of the software, i.e., not worried about where inputs + came from, just worried about computing something from those inputs. + +## Modularity within NWChemEx + +- Introduce module and plugin terminology. + +## Developing a plugin + +## Adding a module to a plugin + +## Next Steps + + diff --git a/docs/_pages/home.md b/docs/_pages/home.md index 8401282..b178046 100644 --- a/docs/_pages/home.md +++ b/docs/_pages/home.md @@ -37,8 +37,14 @@ gallery2: This website is under heavy construction at the moment. {: .notice--warning} -Choose your experience. Don't know which experience you want? Visit our tutorial -on [NWChemEx Community Roles](/tutorials/community_overview). +New to NWChemEx? Check out our [about page](/about/) for a quick overview of the +project and what we are trying to accomplish. Otherwise you can browse +information by use case: + +- **Community members** run calculations. +- **Developers** develop modular software compatible with NWChemEx. +- **Authors** develop infrastructure and plugins for NWChemEx. + {% include gallery id="gallery1" %} From a8ff34b20d3fd4a2aab14c0b298408446395f019 Mon Sep 17 00:00:00 2001 From: "Ryan M. Richard" Date: Tue, 15 Oct 2024 22:21:26 -0500 Subject: [PATCH 2/4] back up [skip ci] --- docs/_pages/developer/home.md | 6 +- docs/_pages/developer/overview.md | 132 ++++++++++++++++++--- docs/_pages/developer/your_first_module.md | 8 ++ docs/_pages/developer/your_first_plugin.md | 8 ++ 4 files changed, 137 insertions(+), 17 deletions(-) create mode 100644 docs/_pages/developer/your_first_module.md create mode 100644 docs/_pages/developer/your_first_plugin.md diff --git a/docs/_pages/developer/home.md b/docs/_pages/developer/home.md index b1a0078..a6df0f8 100644 --- a/docs/_pages/developer/home.md +++ b/docs/_pages/developer/home.md @@ -15,4 +15,8 @@ most useful if you are: and you want the library, tool, or software package to be readily usable by the both the NWChemEx ecosystem and the broader community. - \ No newline at end of file +# Table of Contents + +- [Overview of NWChemEx Development](/developer/overview/) +- [Writing your first NWChemEx Plugin](/developer/your_first_plugin/) +- [Writing your first NWChemEx Module](/developer/your_first_module/) \ No newline at end of file diff --git a/docs/_pages/developer/overview.md b/docs/_pages/developer/overview.md index db06275..86134c4 100644 --- a/docs/_pages/developer/overview.md +++ b/docs/_pages/developer/overview.md @@ -1,35 +1,135 @@ --- title: "Overview of NWChemEx Development" -layout: page +layout: splash permalink: /developer/overview/ --- The NWChemEx ecosystem is predicated on the assumption that developers are -interested in creating and consuming modular software. We begin with an -overview of what modular software is. From there we sketch out the high-level -steps needed to create a plugin or extend an existing plugin. The contents of -this page are conceptual links to concrete tutorials are provided throughout. +interested in creating and consuming modular software. Our purpose here is +to provide enough conceptual background so that developers can understand +what modules are and how modules interact with the rest of the NWChemEx +ecosystem. From there developers will be ready to begin learning how to write +their first module. -## What is a modular software package? +## What is a module? -- Idea is to explain what a modular software package looks like and to - introduce the basic considerations of designing such a package. -- Key point: achieved by worrying about composing quantities, and not how the - quantity is computed. +In software engineering, modules are defined as reusable, single-purpose units +of code that encapsulate their implementation details. The key aspects of this +definition are: -## What is a software module? +- *Reusable.* A piece of software is reusable if it can be used from more than + one place. +- *Single Purpose.* A module should do one task. To do multiple tasks use + multiple modules. +- *Encapsulate.* Modules should be black-boxes, i.e., the caller of the module + should not care how the module does what it does. -- Introduce as opposite of the software, i.e., not worried about where inputs - came from, just worried about computing something from those inputs. +There are also several corollary properties that follow from this definition: + +- *Generalization.* A module must represent some sort of generalization. By + contrast, if a code unit could only be used in a very specific situation it + would not be reusable. +- *Discoverable.* Again follows from the reusable property. If we are to call a + module from multiple places it must be possible for each of those places + to find the module. +- *Composable.* Each module performs one task. Usually those tasks are much + simpler than the user's goal. We thus must be able to use results from + multiple modules to achieve the goal. + +Ultimately the concept of a module provides a mechanism for managing the +complexity of a problem. More specifically, modules allow us to solve new +problems by building on solved problems. For example, say we are trying to sort +a list. Since modules for comparing elements of the list likely exist we can +implement our algorithm in terms of such modules (if the modules did not exist +we could write them, or we could hard-code the the comparison into the module +for the sorting algorithm). Similarly, if we were writing an algorithm +for finding the set of elements common to two lists, we will benefit from the +module for sorting a list. + +## What is modular software? + +At face value modular software is simply software which relies on modules. In +practice almost all modern software is modular at some level; for example, it +probably uses functions, classes, and libraries to break the code up as +opposed to dumping everything into one giant main function. The real difference +between modular and non-modular software is what is called "cohesion." Cohesion +is the extent to which things bundled in a module belong together. Modules where +everything "belongs together" are said to have "high-cohesion" whereas those +which only somewhat belong together are said to have "low-cohesion". Software +made of many high-cohesion modules is termed modular software whereas that +relying on many low-cohesion modules is usually monolithic. + +Why does this matter? Decades of software engineering experience has shown that +it is much harder to maintain, extend, and reuse low-cohesion modules. Moreover, +software formed from low-cohesion modules tends to be more buggy, harder to +use, and harder to extend. This is because modifying low-cohesion modules is +a "game of whack-a-mole." Since low-cohesion modules bring together many +loosely related pieces, changes to those pieces can easily break "distant" +pieces of the software. By contrast, changes to high-cohesion modules tend to +only have ramifications in the module. ## Modularity within NWChemEx -- Introduce module and plugin terminology. +The fundamental premise of the NWChemEx ecosystem is that developers want +to develop high-performance, high-cohesion modules and that software +should be designed in a way that capitalizes on such modules. To that end it is +important to note that some modules, like those implementing +optimization algorithms, can be used in many different circumstances. Similarly, +for many problems, there are often a number of different ways to get the same +solution (again optimization is a good example). Continuing with the optimizer +example, one can imagine that users may have personal preferences for which +optimizer module to use where and/or want to immediately used a bleeding-edge +module wherever optimization is needed. NWChemEx solves this by relying on +dynamic loading of modules, i.e., the decision of which modules to use where +is put off until the software is run. + +The NWChemEx ecosystem is comprised of plugins and +[SimDE](https://github.com/NWChemEx/SimDE). Plugins are libraries +of modules which can be loaded dynamically and SimDE is the software development +kit that contains the infrastructure for loading plugins as well as the +fundamental data structures and APIs needed to interface with plugins. In +practice, this means "streamlined" NWChemEx calculations follow a pattern like: + +```python +import simde +import name_of_plugin + +# mm is an object capable of loading plugins +mm = simde.pluginplay.ModuleManager() + +# Load the plugin using mm +name_of_plugin.load_modules(mm) -## Developing a plugin +# Use mm to run your calculation +``` -## Adding a module to a plugin +We say streamlined because most users won't run NWChemEx this way, rather they +will do: + +```python +import nwchemex as nwx + +mm = nwx.load_modules() + +# Usse mm to run your calculation +``` + +While this may be simpler from a code perspective this loads most of the +plugins in the NWChemEx ecosystem and is significantly heavier in terms of +dependencies. ## Next Steps +To start developing you will need to install SimDE and/or NWChemEx. Strictly +speaking plugins can be created with SimDE alone; however, many developers find +it easier to develop plugins if they can test them using other plugins in the +ecosystem. See [here](/community/install/) for build instructions. + +If you want to learn how to develop a plugin see +[this](/developer/your_first_plugin/) tutorial. + +If you are contributing modules to an already existing plugin see +[this](/developer/your_first_module/) tutorial. + + diff --git a/docs/_pages/developer/your_first_module.md b/docs/_pages/developer/your_first_module.md new file mode 100644 index 0000000..9df58ae --- /dev/null +++ b/docs/_pages/developer/your_first_module.md @@ -0,0 +1,8 @@ +--- +title: "Writing your first NWChemEx Module" +layout: splash +permalink: /developer/your_first_module/ +toc: true +--- + +TODO: Write me!!! \ No newline at end of file diff --git a/docs/_pages/developer/your_first_plugin.md b/docs/_pages/developer/your_first_plugin.md new file mode 100644 index 0000000..52a5f88 --- /dev/null +++ b/docs/_pages/developer/your_first_plugin.md @@ -0,0 +1,8 @@ +--- +title: "Writing your first NWChemEx Plugin" +layout: splash +permalink: /developer/your_first_plugin/ +toc: true +--- + +TODO: Write me!!! \ No newline at end of file From 52e256885175caba0c8b9ed05eaeb3c468b14ec6 Mon Sep 17 00:00:00 2001 From: "Ryan M. Richard" Date: Fri, 15 Nov 2024 22:10:44 -0600 Subject: [PATCH 3/4] backup tutorial --- docs/_pages/developer/your_first_plugin.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/_pages/developer/your_first_plugin.md b/docs/_pages/developer/your_first_plugin.md index 52a5f88..b119485 100644 --- a/docs/_pages/developer/your_first_plugin.md +++ b/docs/_pages/developer/your_first_plugin.md @@ -5,4 +5,16 @@ permalink: /developer/your_first_plugin/ toc: true --- -TODO: Write me!!! \ No newline at end of file +To make life easy the NWChemEx team has created a plugin template + +.. code-block:: .py + + # If cookiecutter is not already installed + pip install cookiecutter + cookiecutter https://github.com/NWChemEx/PluginTemplate.git + +You will then be prompted to answer two questions: + +1. What is the name of project? This should be a human-readable stylized name. +2. What is the name of the project slug? This should be a filesystem-friendly + name. \ No newline at end of file From a922654513eaedfbc35276e3bd7c1da816eb1910 Mon Sep 17 00:00:00 2001 From: "Ryan M. Richard" Date: Sun, 17 Nov 2024 22:30:17 -0600 Subject: [PATCH 4/4] defer to plugin template's instructions --- docs/_pages/developer/your_first_plugin.md | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/docs/_pages/developer/your_first_plugin.md b/docs/_pages/developer/your_first_plugin.md index b119485..4c48d52 100644 --- a/docs/_pages/developer/your_first_plugin.md +++ b/docs/_pages/developer/your_first_plugin.md @@ -5,16 +5,8 @@ permalink: /developer/your_first_plugin/ toc: true --- -To make life easy the NWChemEx team has created a plugin template - -.. code-block:: .py - - # If cookiecutter is not already installed - pip install cookiecutter - cookiecutter https://github.com/NWChemEx/PluginTemplate.git - -You will then be prompted to answer two questions: - -1. What is the name of project? This should be a human-readable stylized name. -2. What is the name of the project slug? This should be a filesystem-friendly - name. \ No newline at end of file +To make life easy the NWChemEx team has created a plugin template which can be +obtained by using the +[Cookiecutter](https://cookiecutter.readthedocs.io/en/stable/) python +package. Instructions for obtaining and configuring the template can be +found [here](https://github.com/NWChemEx/PluginTemplate). \ No newline at end of file