MO-SMAC merge #1222
Open
MO-SMAC merge #1222
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a safety measure. Normally, everytime we update the runhistory, the objective bounds are updated so that the value to normalize should be inside the bound.
Created a helper method to create a set with preserved order from a list
Previously: random scalarization of MO costs bc ParEGO was the only MO algo. Now: separate function which can be overwritten.
Also, reset obtain to the kwargs (no magic numbers) Better debug message
Updates incumbents of runhistory automatically if updated
# Conflicts: # smac/acquisition/maximizer/local_search.py
jeroenrook
commented
Mar 27, 2025
| objectives="accuracy", | ||
| # min_budget=1, # Train the MLP using a hyperparameter configuration for at least 5 epochs | ||
| # max_budget=25, # Train the MLP using a hyperparameter configuration for at most 25 epochs | ||
| n_workers=4, |
| configs_acq.sort(reverse=True, key=lambda x: x[0]) | ||
| for a, inc in configs_acq: | ||
| inc.origin = "Acquisition Function Maximizer: Local Search" | ||
| inc.origin = "Local Search" |
There was a problem hiding this comment.
Fixed this already to make the pytests work again
| def _create_sort_keys(self, costs: np.array) -> list[list[float]]: | ||
| """Non-Dominated Sorting of Costs | ||
|
|
||
| In case of the predictive model returning the prediction for more than one objective per configuration |
There was a problem hiding this comment.
Update text to comply with workings of function
|
|
||
| return init_points | ||
|
|
||
| def _create_sort_keys(self, costs: np.array) -> list[list[float]]: |
There was a problem hiding this comment.
This is to get points to perform the local search with based on earlier runs. So it can make a difference. This code was not implemented by me btw. Only moved probably
| How many incumbents to keep track of in the case of multi-objective. | ||
| """ | ||
| return Intensifier( | ||
| class NewIntensifier(intermediate_decision.NewCostDominatesOldCost, |
There was a problem hiding this comment.
Combine with abstraction of the intensifier
| config_hash = get_config_hash(config) | ||
|
|
||
| # Do not compare very early in the process | ||
| if len(config_isb_keys) < 4: |
There was a problem hiding this comment.
Empirically. But the mixing is likely to overwrite this function anyway
| isb_keys = self.get_incumbent_instance_seed_budget_keys(compare=True) | ||
|
|
||
| n_samples = 1000 | ||
| if len(isb_keys) < 7: # When there are only a limited number of trials available we run all combinations |
There was a problem hiding this comment.
I believe 7 is the miminum number you need to have a least 1000 distinct samples.
# Conflicts: # README.md # smac/acquisition/maximizer/local_search.py # smac/facade/abstract_facade.py # smac/intensifier/abstract_intensifier.py # smac/main/config_selector.py # smac/runner/dask_runner.py
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Multi-objective SMAC as described in https://doi.org/10.1162/evco_a_00371