MO-SMAC merge

README.md

@@ -12,33 +12,12 @@ SMAC offers a robust and flexible framework for Bayesian Optimization to support
 hyperparameter configurations for their (Machine Learning) algorithms, datasets and applications at hand. The main core 
 consists of Bayesian Optimization in combination with an aggressive racing mechanism to efficiently decide which of two configurations performs better.
 
-SMAC3 is written in Python3 and continuously tested with Python 3.8, 3.9, and 3.10. Its Random
-Forest is written in C++. In further texts, SMAC is representatively mentioned for SMAC3.
-
-> [Documentation](https://automl.github.io/SMAC3)
-
-> [Roadmap](https://github.com/orgs/automl/projects/5/views/2)
-
-
-## Important: Changes in v2.0
-
-With the next big major release of SMAC, we drastically boosted the user experience by improving the APIs and how the 
-pipelining is done (see [changelog](CHANGELOG.md)). All facades/intensifiers support multi-objective, multi-fidelity, 
-and multi-threading natively now! That includes having an ask-and-tell interface and continuing a run
-wherever you left off. pSMAC is removed because when specifying the number of workers, SMAC automatically uses 
-multi-threading for evaluating trials. When cleaning the code base, however, we removed the command-line 
-interface (calling a target function from a script is still supported), and runtime optimization. Also,
-python 3.7 is not supported anymore. If you depend on those functionalities, please keep using v1.4.
-
-We are excited to introduce the new major release and look forward to developing new features on the new code base. 
-We hope you enjoy this new user experience as much as we do. 🚀
-
+MO-SMAC is implemented directly into SMAC3. This repository is forked from the [SMAC3 repository](https://github.com/automl/SMAC3) and therefore contains references and copyright information to those authors. 
+These do not align with the authors of MO-SMAC and, therefore, the anonymity for this repository remains intact.
 
 ## Installation
 
-This instruction is for the installation on a Linux system, for Windows and Mac and further information see the [documentation](https://automl.github.io/SMAC3/main/1_installation.html).
-
-Create a new environment with python 3.10 and make sure swig is installed either on your system or
+Create a new environment with Python 3.10 and make sure swig is installed either on your system or
 inside the environment. We demonstrate the installation via anaconda in the following:
 
 Create and activate environment:
@@ -52,25 +31,27 @@ Install swig:
 conda install gxx_linux-64 gcc_linux-64 swig
 ```
 
-Install SMAC via PyPI:
-```
-pip install smac
+Clone this repository and install locally:
 ```
 
 If you want to contribute to SMAC, use the following steps instead:
 ```
 git clone https://github.com/automl/SMAC3.git && cd SMAC3
 make install-dev
+cd SMAC3K
+pip install -e .[dev]
 ```
 
-
 ## Minimal Example
+To use MO-SMAC, there is a multi-objective facade that provides all the functionalities for MO-AAC. The example below shows how this facade can be accessed and used. 
 
 ```py
 from ConfigSpace import Configuration, ConfigurationSpace
 
+import time 
 import numpy as np
-from smac import HyperparameterOptimizationFacade, Scenario
+from smac.facade.multi_objective_facade import MultiObjectiveFacade
+from smac import Scenario
 from sklearn import datasets
 from sklearn.svm import SVC
 from sklearn.model_selection import cross_val_score
@@ -80,68 +61,21 @@ iris = datasets.load_iris()
 
 def train(config: Configuration, seed: int = 0) -> float:
     classifier = SVC(C=config["C"], random_state=seed)
+    start_time = time.time()
     scores = cross_val_score(classifier, iris.data, iris.target, cv=5)
-    return 1 - np.mean(scores)
+    run_time = time.time() - start_time
+    return {"perf": 1 - np.mean(scores), "runtime": run_time}
 
 
 configspace = ConfigurationSpace({"C": (0.100, 1000.0)})
 
 # Scenario object specifying the optimization environment
-scenario = Scenario(configspace, deterministic=True, n_trials=200)
+scenario = Scenario(configspace, 
+                    deterministic=True, 
+                    n_trials=200,
+                    objectives=["perf", "runtime"])
 
 # Use SMAC to find the best configuration/hyperparameters
-smac = HyperparameterOptimizationFacade(scenario, train)
+smac = MultiObjectiveFacade(scenario, train)
 incumbent = smac.optimize()
 ```
-
-More examples can be found in the [documentation](https://automl.github.io/SMAC3/main/examples/).
-
-## Visualization via DeepCAVE
-
-With DeepCAVE ([Repo](https://github.com/automl/DeepCAVE), [Paper](https://arxiv.org/abs/2206.03493)) you can visualize your SMAC runs. It is a visualization and analysis tool for AutoML (especially for the sub-problem
-hyperparameter optimization) runs.
-
-## License
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the 3-clause BSD license (please see the LICENSE file).
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-
-You should have received a copy of the 3-clause BSD license
-along with this program (see LICENSE file).
-If not, see [here](https://opensource.org/licenses/BSD-3-Clause).
-
-## Contacting us
-
-If you have trouble using SMAC, a concrete question or found a bug, please create an [issue](https://github.com/automl/SMAC3/issues). This is the easiest way to communicate about these things with us. 
-
-For all other inquiries, please write an email to smac[at]ai[dot]uni[dash]hannover[dot]de.
-
-## Miscellaneous
-
-SMAC3 is developed by the [AutoML Groups of the Universities of Hannover and
-Freiburg](http://www.automl.org/).
-
-If you have found a bug, please report to [issues](https://github.com/automl/SMAC3/issues). Moreover, we are 
-appreciating any kind of help. Find our guidelines for contributing to this package 
-[here](CONTRIBUTING.md).
-
-If you use SMAC in one of your research projects, please cite our 
-[JMLR paper](https://jmlr.org/papers/v23/21-0888.html):
-```
-@article{JMLR:v23:21-0888,
-  author  = {Marius Lindauer and Katharina Eggensperger and Matthias Feurer and André Biedenkapp and Difan Deng and Carolin Benjamins and Tim Ruhkopf and René Sass and Frank Hutter},
-  title   = {SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization},
-  journal = {Journal of Machine Learning Research},
-  year    = {2022},
-  volume  = {23},
-  number  = {54},
-  pages   = {1--9},
-  url     = {http://jmlr.org/papers/v23/21-0888.html}
-}
-```
-
-Copyright (C) 2016-2022  [AutoML Group](http://www.automl.org).

examples/2_multi_fidelity/1_mlp_epochs.py

@@ -80,7 +80,7 @@ def configspace(self) -> ConfigurationSpace:
 
         return cs
 
-    def train(self, config: Configuration, seed: int = 0, budget: int = 25) -> float:
+    def train(self, config: Configuration, seed: int = 0, instance: str = "0", budget: int = 25) -> dict[str, float]:
         # For deactivated parameters (by virtue of the conditions),
         # the configuration stores None-values.
         # This is not accepted by the MLP, so we replace them with placeholder values.
@@ -106,7 +106,7 @@ def train(self, config: Configuration, seed: int = 0, budget: int = 25) -> float
             cv = StratifiedKFold(n_splits=5, random_state=seed, shuffle=True)  # to make CV splits consistent
             score = cross_val_score(classifier, dataset.data, dataset.target, cv=cv, error_score="raise")
 
-        return 1 - np.mean(score)
+        return {"accuracy": 1 - np.mean(score)}
 
 
 def plot_trajectory(facades: list[AbstractFacade]) -> None:
@@ -147,9 +147,11 @@ def plot_trajectory(facades: list[AbstractFacade]) -> None:
             mlp.configspace,
             walltime_limit=60,  # After 60 seconds, we stop the hyperparameter optimization
             n_trials=500,  # Evaluate max 500 different trials
-            min_budget=1,  # Train the MLP using a hyperparameter configuration for at least 5 epochs
-            max_budget=25,  # Train the MLP using a hyperparameter configuration for at most 25 epochs
-            n_workers=8,
+            instances=[str(i) for i in range(10)],
+            objectives="accuracy",
+            # min_budget=1,  # Train the MLP using a hyperparameter configuration for at least 5 epochs
+            # max_budget=25,  # Train the MLP using a hyperparameter configuration for at most 25 epochs
+            n_workers=4,
         )
 
         # We want to run five random configurations before starting the optimization.

smac/acquisition/function/abstract_acquisition_function.py

@@ -50,7 +50,7 @@ def update(self, model: AbstractModel, **kwargs: Any) -> None:
 
         This method will be called after fitting the model, but before maximizing the acquisition
         function. As an examples, EI uses it to update the current fmin. The default implementation only updates the
-        attributes of the acqusition function which are already present.
+        attributes of the acquisition function which are already present.
 
         Calls `_update` to update the acquisition function attributes.
 
@@ -65,7 +65,7 @@ def update(self, model: AbstractModel, **kwargs: Any) -> None:
         self._update(**kwargs)
 
     def _update(self, **kwargs: Any) -> None:
-        """Update acsquisition function attributes
+        """Update acquisition function attributes
 
         Might be different for each child class.
         """