[Feature Request] Support Optuna 3.x and 4.x in hydra-optuna-sweeper

[Northo]

🚀 Feature Request

Optuna is currently constrained to <3.0.0 in

hydra/plugins/hydra_optuna_sweeper/setup.py

Line 30 in ca4d25c

"optuna>=2.10.0,<3.0.0",

It would be great to support the latest Optuna version, 4.1.0.

#2360 started bumping to 3.0.0, but it was never merged.

Motivation

Is your feature request related to a problem? Please describe.

This constraint makes it inconvenient to add hydra-optuna-sweeper in projects where newer versions of Optuna is already used, or when one needs some of the newer features from Optuna

Pitch

Describe the solution you'd like

Support Optuna version 4.1.0.

Describe alternatives you've considered

The alternative is to either restrict oneself to <3.0.0, or not use the plugin.

Are you willing to open a pull request? (See CONTRIBUTING)
No

[Northo]

I believe this is a duplicate of #2162, sorry.