Merge pull request #613 from BlockScience/0.4.0

V0.4.0

Author: SeanMcOwen

dev/Issue Matrix.ipynb

@@ -12,6 +12,7 @@ As part of MSML, there are multiple canonical examples which are meant to showca
 | Feature/Attribute | [MSML Template](https://github.com/BlockScience/MSML-Template) | [Predator Prey](https://github.com/BlockScience/Predator-Prey-MSML) | [Root Finding](https://github.com/SeanMcOwen/Root-Finding-Simulation) | [Retirement Planning](https://github.com/BlockScience/Retirement-Planning-MSML) |
 | --- | --- | --- | --- | --- |
 | Description | A template for using MSML. It implements very basic functionality. | An example of building out an agent based simulation for modeling predators and prey based on multiple actions. | A simulation of different root finding techniques and their relative speed/accuracy | A canonical example to showcase how to simulate changing behavioral patterns and their impacts on final results|
+| Complexity (1-5) | 1 | 4 | 3 | 2 |
 | Spec Implementation Complete? | X | X | WIP | WIP |
 | Code Implementations Complete? | X | X |  | WIP |
 | Simulations/Experiments Complete? | X | WIP|  | |

docs/CanonicalExamples.md

@@ -37,13 +37,7 @@ Writing mathematical specifications can be a difficult process, especially when
 ## How does MSML work?
 
 
-```mermaid
-graph TD
-A[JSON Object \n\n Each spec has a repo for tracking changes \n Must conform to the json specification \n Defines all aspects of the spec including blocks, spaces and actions] -->B[MSML Object \n\n JSON file is parsed, with validations and mappings along the way \n Can show different views on the fly]
-    B --> C[Report Outputs & Obsidian Directory \n\n Automatically build reports for the full spec or subviews \n Example: all blocks with an effect on variable XYZ\n Also builds an entire Obsidian directory of all components as linked notes]
-D[Python Function Implementations \n\n Optional enhancement to actually execute code\n Done for each referenced policy option, mechanism, etc. \n Just needs a function definition for each] --> B 
-B --> E[Python Wirings & Simulations \n\n MSML can be used to run blocks \n Wirings automatically work to pass between domain/codomains \n Entire simulations can be built up as composed wirings]
-```
+![mermaid](mermaid.png)
 
 ## MSML in the Engineering Lifecycle
 

docs/GettingStarted.md

@@ -3,7 +3,7 @@ requires = ["setuptools>=61.0"]
 build-backend = "setuptools.build_meta"
 [project]
 name = "math-spec-mapping"
-version = "0.3.21"
+version = "0.4.0"
 authors = [
   { name="Sean McOwen", email="[email protected]" },
 ]

docs/index.md

@@ -1,45 +1,83 @@
-# MSML v0.3 Update
+# MSML v0.4 Update
 
-Updated through V0.3.6
+Updated through V0.3.14
 
 ## MSML Review
 
 MSML is a library for standardizing the creation of mathematical specifications as JSON objects as well as aiding in the automation of report and visualization creation from these standardized JSON. Please see the [README](../README.md) for more information.
 
 ## Updates from to v0.3.0 to V0.4.0
 
-A total of XX issues were closed out over the development cycle, which broadly fall into the following categories:
+A total of XX issues were closed out over the development cycle, which broadly fall into the following categories below. However, for brevity, only the main updates will be covered, many smaller updates such as bug fixes, small improvements, etc. will not be noted here but can be found in the research journal.
 
 ### Implementations / Executable Code Blocks
 
 - Boundary actions are now added in as an implementation
 - Full wiring runs are now working/executing
 - New functionality for validation of state and params before running wiring or blocks
 - Stateful metrics were implemented and now are passed in as a module in the state that implementations can access and use
+- State and parameter preparation function paradigms were added into MSML to allow for setting up parameters and state prior to simulations / executable codes
+- Metrics were added into implementations to allow for mapping code to them
+- Functionality was added in for linking source code to the blocks for easy viewing and writing into markdown reports; as well a source code printing function was added
+- An enhanced version of the components running was created which acts as a wrapper around executable code and checks things like the actual spaces match even when doing the code execution side
 
 ### Metaprogramming
 
 - Initial R&D into metaprogramming for Julia has yielded some success on metaprogramming for spaces and types in cadCAD.jl from MSML
 - Boundary action metaprogramming has been started
 
+### Simulations
+
+- The mini-simulation runner was created
+- Post-processing function paradigm was added into the mini-simulation runner
+- Metric functions were added to the runner for post processing of data
+- Ability to run lists of blocks built out
+- Mapping of parameter changes to columns for experiments was added in
+
 ### Canonical Examples
 
+- The MSML template was added as both a canonical example as well as a forkable template for getting started quickly with MSML
+- The predator prey canonical example was created and has been completed up to simulations
 - Introduction of the retirement planning canonical example as a guided walk through example
 - Introduction of the Rideshare Modeling canonical example for building a massive end-to-end model of a rideshare simulation
-- Big improvements to the starter repo including adding in executable code blocks and better naming
+- Big improvements to the starter repo including adding in executable code blocks and better naming, which was later rolled into the MSML template
 
 ## Research Notes
 
 - A research note on metrics was created and worked through with stakeholders
 - A research note on parameter class names and different ways that it could be implemented (plan is for a "bring your own ontology" approach moving forward)
 - Development note on working through an issue with Latex escape characters
 - Development research note for future proposals of changes to MSML
+- Predator-Prey-Ideation through reverse engineering research note
+- Multi-processing research note for aiding with cadCAD performance (and eventual dogfooding on MSML)
+- V0.5 release plan was created
+- V0.4 update note was created
 
 ## Improvements and Convenience Functions
 
 - A lot of bugs were discovered and fixed through dogfooding on client work and canonical examples
 - Some major improvements across the board in terms of quality of life stuff; a lot of new assertions that are more descriptive added to easily see where the issues are
 - New convenience functions such as one to remove dummy repo components after the starter repo scaffolding is no longer needed
+- A parameter table creation function was built which automatically creates a markdown table of parameters, descriptions, and default values for experiment notebooks
+- Improvements of assertions
+- An automatic README writer was added in
+- A functionality that automatically writes issues from an MSML scaffold into a github issues was built out
+- Large amount of small updates to the reporting sections
+- Functionality was added which puts the actual bound code into the markdown reports and also adds links to both the definitions of code as well as the spec component definitions for making it easier to develop
+- Depth of 1 mermaid wiring charts were created to make it cleaner to look at different wirings
+- The nodes in mermaid graphics are now clickable and bring you to the Obsidian note for that component
+- Automated github table creation functionality for project management improvement
+- Full issue inventory and tagging of all 200 issues within the MSML repository
+- The remove starter repo functionality was refactored for the new MSML template
+- The official [MSML documentation](https://blockscience.github.io/MSML/) has been launched
+- A getting started guide was built
+
+## Presentations
+
+- The annual presentation was given in June 2024 during the town hall
+- A governance pod presentation titled "Obsidian, the Mathematical Specification Mapping Library, and End-to-End Modeling" was given
+- October seminar presentation
+- A research note on Predator-Prey-Ideation through reverse engineering was completed which shows how to turn legacy code into an MSML scaffold then automatically port it into github issues for easy organization of the next step (the actual MSML spec), presented in seminar
 
 ## Future Research Arc
 

docs/mermaid.png

@@ -727,11 +727,13 @@ def run_experiments(
         state_preperation_functions=None,
         parameter_preperation_functions=None,
         metrics_functions=None,
+        append_param_change_columns=False,
     ):
         state_l = []
         params_l = []
         df_l = []
         metrics_l = []
+        param_mods = set()
         for experiment in experiments:
             for i in range(experiment["Monte Carlo Runs"]):
                 state, params, msi, df, metrics = self.run_experiment(
@@ -743,6 +745,11 @@ def run_experiments(
                     parameter_preperation_functions=parameter_preperation_functions,
                     metrics_functions=metrics_functions,
                 )
+                if append_param_change_columns:
+                    if experiment["Param Modifications"]:
+                        for key in experiment["Param Modifications"]:
+                            df[key] = experiment["Param Modifications"][key]
+                            param_mods.add(key)
                 df["Monte Carlo Run"] = i + 1
                 df["Experiment"] = experiment["Name"]
                 metrics.loc["Monte Carlo Run"] = i + 1
@@ -753,6 +760,9 @@ def run_experiments(
                 df_l.append(df)
                 metrics_l.append(metrics)
         df = pd.concat(df_l)
+        if append_param_change_columns:
+            for key in param_mods:
+                df[key] = df[key].fillna(params_base[key])
         metrics = pd.concat(metrics_l, axis=1).T
         return df, metrics, state_l, params_l
 
@@ -1022,6 +1032,12 @@ def load_control_actions(self):
                     ), "No functional parameterization for {}. To fix this error, add {} to the parameters passed to ms.build_implementation. Option can be: {}".format(
                         ca.name, "FP " + ca.name, [x.name for x in opts]
                     )
+                    assert (
+                        self.params["FP {}".format(ca.name)]
+                        in self.ms.functional_parameters["FP {}".format(ca.name)]
+                    ), "{} is not a valid functional parameterization for {}".format(
+                        self.params["FP {}".format(ca.name)], ca.name
+                    )
                     opt = self.ms.functional_parameters["FP {}".format(ca.name)][
                         self.params["FP {}".format(ca.name)]
                     ]
@@ -1061,6 +1077,13 @@ def load_boundary_actions(self):
                         ba.name, "FP " + ba.name, [x.name for x in opts]
                     )
 
+                    assert (
+                        self.params["FP {}".format(ba.name)]
+                        in self.ms.functional_parameters["FP {}".format(ba.name)]
+                    ), "{} is not a valid functional parameterization for {}".format(
+                        self.params["FP {}".format(ba.name)], ba.name
+                    )
+
                     opt = self.ms.functional_parameters["FP {}".format(ba.name)][
                         self.params["FP {}".format(ba.name)]
                     ]
@@ -1165,6 +1188,12 @@ def load_policies(self):
                     ), "No functional parameterization for {}. To fix this error, add {} to the parameters passed to ms.build_implementation. Option can be: {}".format(
                         p.name, "FP " + p.name, [x.name for x in opts]
                     )
+                    assert (
+                        self.params["FP {}".format(p.name)]
+                        in self.ms.functional_parameters["FP {}".format(p.name)]
+                    ), "{} is not a valid functional parameterization for {}".format(
+                        self.params["FP {}".format(p.name)], p.name
+                    )
                     opt = self.ms.functional_parameters["FP {}".format(p.name)][
                         self.params["FP {}".format(p.name)]
                     ]

pyproject.toml

@@ -92,6 +92,15 @@ def create_milestone_label_matrix(df, exclude_milestones=None):
     labels = list(df.columns[4:])
     labels = sorted([x for x in labels if x not in priority_labels])
     milestones = sorted(list(df[~pd.isnull(df["Milestone"])]["Milestone"].unique()))
+    try:
+        milestones = sorted(
+            milestones,
+            key=lambda x: int(x.split(".")[0][1:]) * 10000
+            + int(x.split(".")[1]) * 100
+            + int(x.split(".")[2]),
+        )
+    except:
+        pass
 
     table = []
     for label in labels: