Skip to content

Latest version of FLAML changes optimization behavior #839

Open
@bpkroth

Description

@bpkroth

Recently a new version (v2.2.0) of FLAML was released and it introduces some new warnings and behavior:

INFO     flaml.tune.searcher.blendsearch:blendsearch.py:169 No low-cost partial config given to the search algorithm. For cost-frugal search, consider providing low-cost values for cost-related hps via 'low_cost_partial_config'. More info can be found at https://microsoft.github.io/FLAML/docs/FAQ#about-low_cost_partial_config-in-tune
WARNING  flaml.tune.searcher.suggestion:suggestion.py:555 You passed a `space` parameter to OptunaSearch that contained unresolved search space definitions. OptunaSearch should however be instantiated with fully configured search spaces only. To use Ray Tune's automatic search space conversion, pass the space definition as part of the `config` argument to `tune.run()` instead.
WARNING  flaml.tune.searcher.suggestion:suggestion.py:555 You passed a `space` parameter to OptunaSearch that contained unresolved search space definitions. OptunaSearch should however be instantiated with fully configured search spaces only. To use Ray Tune's automatic search space conversion, pass the space definition as part of the `config` argument to `tune.run()` instead.
FAILED mlos_bench/mlos_bench/tests/optimizers/toy_optimization_loop_test.py::test_flaml_optimization_loop - assert 63.77786073141286 == 60.15 ± 6.0e-01
  
  comparison failed
  Obtained: 63.77786073141286
  Expected: 60.15 ± 6.0e-01

We should investigate and fix these.

For now, I'm creating a PR to simply pin the version to avoid them until we have time to do that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions