[WIP] release(4.0.0): OpenSTEF 4.0.0 tracking branch.#637
Open
[WIP] release(4.0.0): OpenSTEF 4.0.0 tracking branch.#637
Conversation
Signed-off-by: Egor Dmitriev <[email protected]>
* feature(STEF-2297): Initial commit. Cleared OpenSTEF 3.0 code. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Updated readme to exclude 3.0 specific parts. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Created directory structure for the 4.0. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Set python version to 3.12 Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(STEF-2297): Initial commit. Cleared OpenSTEF 3.0 code. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Updated readme to exclude 3.0 specific parts. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Created directory structure for the 4.0. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Configured Reuse tool. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Created reuse fix tool. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Added initial ruff config. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Configured ruff, poe, pytest and pyright. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Configured pyproject-fmt. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Added example scripts for ci testing. Added example integration test. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Added support for doctests. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(STEF-2297): Initial commit. Cleared OpenSTEF 3.0 code. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Updated readme to exclude 3.0 specific parts. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2297): Created directory structure for the 4.0. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Configured Reuse tool. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Created reuse fix tool. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Added initial ruff config. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Configured ruff, poe, pytest and pyright. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Configured pyproject-fmt. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Added example scripts for ci testing. Added example integration test. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2298): Added support for doctests. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added added github action for REUSE compliance check. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added added github quality checks. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Fixed reuse compliance. in gha. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Changed triggers for gha checks. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Changed triggers for gha checks. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Narrowed check gha permissions. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Updated github action. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added coverage checking. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added coverage checking. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added release / dev build job. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Updated checks workflow. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Updated checks workflow. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Updated checks workflow. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Improved version pinning. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added extra and default dependencies for openstef root package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Added extra and default dependencies for openstef root package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Enabled dev release publishing. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Disabled dev release publishing until mvp is ready. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2299): Style fixes. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(STEF-2300): Changed python dev version. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added docs setup. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Fixed styles. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added improved docs and api documentation. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added new docs. Added contributing guide. Updated readme. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Added new docs. Added contributing guide. Updated readme. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Updated issue template links and slack channel mentions. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2300): Updated docs gha trigger to remove test branch. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
Collaborator
Author
Signed-off-by: Egor Dmitriev <[email protected]>
…added documentation on how to do logging. (#643) * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#629): Added logging configuration and documentation on how to use logging in this package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#629): Added license gitignore exclusion for vscode stuff. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-compatibility/README.md Co-authored-by: Lars Schilders <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * change: Updated docs. Removed vscode example config. * change: Added all modules to api reference. Removed context lib import from init. * feature: Sign commit. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Lars Schilders <[email protected]>
…red code. (#645) * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Copilot <[email protected]>
* feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Improved reference docs. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(#628): Migrated openstef beam backtesting. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Migrated openstef beam backtesting. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Migrated metrics to openstef beam. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Migrated metrics and metric provider. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Migrated evaluation code. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Migrated analysis module to openstef beam. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Migrated benchmarking module to openstef beam. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added module level docs. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added docs for backtesting, evaluation and core. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added docs for analysis module. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Documented openstef beam benchmark. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added docs for benchmarking module. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated docstrings to avoid references to model interface and be more general regarding forecast targets. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
…rsioned parts to avoid O(n^2) problem when doing cross dataset operations. (#657) * feature(#628): Started on versioned timeseries dataset. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Started on versioned timeseries dataset. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Started on versioned timeseries dataset. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated rest of the code to use new versioned dataset type. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated rest of the code to use new versioned dataset type. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Fixed doctests. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Improved docs. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated docsting in mixin for select function and inherited it from the dataset part. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated tests and removed unnecessary docstrings. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated tests and removed unnecessary docstrings. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* Update CITATION.cff (#642) * Update CITATION.cff order citation according to DOI on zenodo Signed-off-by: Frank Kreuwel <[email protected]> * Update CITATION.cff Signed-off-by: Frank Kreuwel <[email protected]> * fix: yaml formatting Signed-off-by: Bart Pleiter <[email protected]> --------- Signed-off-by: Frank Kreuwel <[email protected]> Signed-off-by: Bart Pleiter <[email protected]> Co-authored-by: Bart Pleiter <[email protected]> * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * add cylic features Signed-off-by: lschilders <[email protected]> * update cylic features Signed-off-by: lschilders <[email protected]> * add cylic features tests Signed-off-by: lschilders <[email protected]> * add timeOfDay cyclic features Signed-off-by: lschilders <[email protected]> * configurable cyclic features Signed-off-by: lschilders <[email protected]> * configurable cyclic features Signed-off-by: lschilders <[email protected]> * add rolling aggregate features Signed-off-by: lschilders <[email protected]> * add rolling aggregate features tests Signed-off-by: lschilders <[email protected]> * add rolling aggregate features tests Signed-off-by: lschilders <[email protected]> * add datetime features and tests Signed-off-by: lschilders <[email protected]> * add feature clipper and tests Signed-off-by: lschilders <[email protected]> * added scaling transform Signed-off-by: lschilders <[email protected]> * added scaling transform tests Signed-off-by: lschilders <[email protected]> * fixed all poe tasks Signed-off-by: lschilders <[email protected]> * add daylight features Signed-off-by: lschilders <[email protected]> * import or skip pvlib Signed-off-by: lschilders <[email protected]> * remove icalendar dependency Signed-off-by: lschilders <[email protected]> * fix test formatting * install all packages in workflow * fix test formatting * feature(#626): Missing value imputation (#649) * Initial implementation for missing values handling feature engineering Signed-off-by: Marnix van Lieshout <[email protected]> * Add to pyproject Signed-off-by: Marnix van Lieshout <[email protected]> * Added logic to remove rows with trailing nan for a feature. Signed-off-by: Marnix van Lieshout <[email protected]> * Add tests. Signed-off-by: Marnix van Lieshout <[email protected]> * Restore pyproject.toml and uv.lock from origin/feature/626-feature-engineering Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting, formatting Signed-off-by: Marnix van Lieshout <[email protected]> * More linting and formatting Signed-off-by: Marnix van Lieshout <[email protected]> * Fix typing Signed-off-by: Marnix van Lieshout <[email protected]> * Review comments Signed-off-by: Marnix van Lieshout <[email protected]> * Pyright Signed-off-by: Marnix van Lieshout <[email protected]> * Changes based on review comments Signed-off-by: Marnix van Lieshout <[email protected]> * Fix tests and warnings Signed-off-by: Marnix van Lieshout <[email protected]> * Fix warnings Signed-off-by: Marnix van Lieshout <[email protected]> * Fix fixtures. Signed-off-by: Marnix van Lieshout <[email protected]> * Clean up Signed-off-by: Marnix van Lieshout <[email protected]> * Add warning assert to test Signed-off-by: Marnix van Lieshout <[email protected]> * Fix attribute type Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting Signed-off-by: Marnix van Lieshout <[email protected]> * Reset uv.lock Signed-off-by: Marnix van Lieshout <[email protected]> * Config as part of init Signed-off-by: Marnix van Lieshout <[email protected]> * Remove type ignores Signed-off-by: Marnix van Lieshout <[email protected]> * Extend transform from baseconfig Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting Signed-off-by: Marnix van Lieshout <[email protected]> * Validate for fill value when constant strategy. Signed-off-by: Marnix van Lieshout <[email protected]> * Use default value for missing_value Signed-off-by: Marnix van Lieshout <[email protected]> * Move imputer to private attribute. Signed-off-by: Marnix van Lieshout <[email protected]> * Fix warning Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]> * initial commit weather transforms Signed-off-by: lschilders <[email protected]> * fix type errors * fix most type and formatting errors * fix all errors except for weather transforms * fix radiation_derived_features tests * vectorize feature clipper computation * rename scaler to scaler_transform * rename rolling aggregate features * formatting Signed-off-by: lschilders <[email protected]> * feature(#628): Update feature naming. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated forecasting transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Refactored temporal transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Refactored temporal transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Fixed tests and doctests. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated docstring to include the right unit. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated doctests to include output examples. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Frank Kreuwel <[email protected]> Signed-off-by: Bart Pleiter <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: lschilders <[email protected]> Signed-off-by: Marnix van Lieshout <[email protected]> Co-authored-by: Frank Kreuwel <[email protected]> Co-authored-by: Bart Pleiter <[email protected]> Co-authored-by: Egor Dmitriev <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Marnix van Lieshout <[email protected]>
* Update CITATION.cff (#642) * Update CITATION.cff order citation according to DOI on zenodo Signed-off-by: Frank Kreuwel <[email protected]> * Update CITATION.cff Signed-off-by: Frank Kreuwel <[email protected]> * fix: yaml formatting Signed-off-by: Bart Pleiter <[email protected]> --------- Signed-off-by: Frank Kreuwel <[email protected]> Signed-off-by: Bart Pleiter <[email protected]> Co-authored-by: Bart Pleiter <[email protected]> * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * add cylic features Signed-off-by: lschilders <[email protected]> * update cylic features Signed-off-by: lschilders <[email protected]> * add cylic features tests Signed-off-by: lschilders <[email protected]> * add timeOfDay cyclic features Signed-off-by: lschilders <[email protected]> * configurable cyclic features Signed-off-by: lschilders <[email protected]> * configurable cyclic features Signed-off-by: lschilders <[email protected]> * add rolling aggregate features Signed-off-by: lschilders <[email protected]> * add rolling aggregate features tests Signed-off-by: lschilders <[email protected]> * add rolling aggregate features tests Signed-off-by: lschilders <[email protected]> * add datetime features and tests Signed-off-by: lschilders <[email protected]> * add feature clipper and tests Signed-off-by: lschilders <[email protected]> * added scaling transform Signed-off-by: lschilders <[email protected]> * added scaling transform tests Signed-off-by: lschilders <[email protected]> * fixed all poe tasks Signed-off-by: lschilders <[email protected]> * add daylight features Signed-off-by: lschilders <[email protected]> * import or skip pvlib Signed-off-by: lschilders <[email protected]> * remove icalendar dependency Signed-off-by: lschilders <[email protected]> * fix test formatting * install all packages in workflow * fix test formatting * feature(#626): Missing value imputation (#649) * Initial implementation for missing values handling feature engineering Signed-off-by: Marnix van Lieshout <[email protected]> * Add to pyproject Signed-off-by: Marnix van Lieshout <[email protected]> * Added logic to remove rows with trailing nan for a feature. Signed-off-by: Marnix van Lieshout <[email protected]> * Add tests. Signed-off-by: Marnix van Lieshout <[email protected]> * Restore pyproject.toml and uv.lock from origin/feature/626-feature-engineering Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting, formatting Signed-off-by: Marnix van Lieshout <[email protected]> * More linting and formatting Signed-off-by: Marnix van Lieshout <[email protected]> * Fix typing Signed-off-by: Marnix van Lieshout <[email protected]> * Review comments Signed-off-by: Marnix van Lieshout <[email protected]> * Pyright Signed-off-by: Marnix van Lieshout <[email protected]> * Changes based on review comments Signed-off-by: Marnix van Lieshout <[email protected]> * Fix tests and warnings Signed-off-by: Marnix van Lieshout <[email protected]> * Fix warnings Signed-off-by: Marnix van Lieshout <[email protected]> * Fix fixtures. Signed-off-by: Marnix van Lieshout <[email protected]> * Clean up Signed-off-by: Marnix van Lieshout <[email protected]> * Add warning assert to test Signed-off-by: Marnix van Lieshout <[email protected]> * Fix attribute type Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting Signed-off-by: Marnix van Lieshout <[email protected]> * Reset uv.lock Signed-off-by: Marnix van Lieshout <[email protected]> * Config as part of init Signed-off-by: Marnix van Lieshout <[email protected]> * Remove type ignores Signed-off-by: Marnix van Lieshout <[email protected]> * Extend transform from baseconfig Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting Signed-off-by: Marnix van Lieshout <[email protected]> * Validate for fill value when constant strategy. Signed-off-by: Marnix van Lieshout <[email protected]> * Use default value for missing_value Signed-off-by: Marnix van Lieshout <[email protected]> * Move imputer to private attribute. Signed-off-by: Marnix van Lieshout <[email protected]> * Fix warning Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]> * initial commit weather transforms Signed-off-by: lschilders <[email protected]> * fix type errors * fix most type and formatting errors * fix all errors except for weather transforms * fix radiation_derived_features tests * Init setup Signed-off-by: Marnix van Lieshout <[email protected]> * vectorize feature clipper computation * rename scaler to scaler_transform * rename rolling aggregate features * formatting Signed-off-by: lschilders <[email protected]> * Added sufficient complete check. Added tests. Signed-off-by: Marnix van Lieshout <[email protected]> * Remove comment Signed-off-by: Marnix van Lieshout <[email protected]> * feature(#628): Update feature naming. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated forecasting transforms. Signed-off-by: Egor Dmitriev <[email protected]> * Fix docs Signed-off-by: Marnix van Lieshout <[email protected]> * More testcases Signed-off-by: Marnix van Lieshout <[email protected]> * Raise error and optionally run on transform Signed-off-by: Marnix van Lieshout <[email protected]> * Add override Signed-off-by: Marnix van Lieshout <[email protected]> * Raise error or return original dataframe Signed-off-by: Marnix van Lieshout <[email protected]> * Fix based on review Signed-off-by: Marnix van Lieshout <[email protected]> * Minor fixes Signed-off-by: Marnix van Lieshout <[email protected]> * Remove state from completeness check Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Frank Kreuwel <[email protected]> Signed-off-by: Bart Pleiter <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: lschilders <[email protected]> Signed-off-by: Marnix van Lieshout <[email protected]> Co-authored-by: Frank Kreuwel <[email protected]> Co-authored-by: Bart Pleiter <[email protected]> Co-authored-by: Egor Dmitriev <[email protected]> Co-authored-by: lschilders <[email protected]> Co-authored-by: Copilot <[email protected]>
* Update CITATION.cff (#642) * Update CITATION.cff order citation according to DOI on zenodo Signed-off-by: Frank Kreuwel <[email protected]> * Update CITATION.cff Signed-off-by: Frank Kreuwel <[email protected]> * fix: yaml formatting Signed-off-by: Bart Pleiter <[email protected]> --------- Signed-off-by: Frank Kreuwel <[email protected]> Signed-off-by: Bart Pleiter <[email protected]> Co-authored-by: Bart Pleiter <[email protected]> * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * add cylic features Signed-off-by: lschilders <[email protected]> * update cylic features Signed-off-by: lschilders <[email protected]> * add cylic features tests Signed-off-by: lschilders <[email protected]> * add timeOfDay cyclic features Signed-off-by: lschilders <[email protected]> * configurable cyclic features Signed-off-by: lschilders <[email protected]> * configurable cyclic features Signed-off-by: lschilders <[email protected]> * add rolling aggregate features Signed-off-by: lschilders <[email protected]> * add rolling aggregate features tests Signed-off-by: lschilders <[email protected]> * add rolling aggregate features tests Signed-off-by: lschilders <[email protected]> * add datetime features and tests Signed-off-by: lschilders <[email protected]> * add feature clipper and tests Signed-off-by: lschilders <[email protected]> * added scaling transform Signed-off-by: lschilders <[email protected]> * added scaling transform tests Signed-off-by: lschilders <[email protected]> * fixed all poe tasks Signed-off-by: lschilders <[email protected]> * add daylight features Signed-off-by: lschilders <[email protected]> * import or skip pvlib Signed-off-by: lschilders <[email protected]> * remove icalendar dependency Signed-off-by: lschilders <[email protected]> * fix test formatting * install all packages in workflow * fix test formatting * feature(#626): Missing value imputation (#649) * Initial implementation for missing values handling feature engineering Signed-off-by: Marnix van Lieshout <[email protected]> * Add to pyproject Signed-off-by: Marnix van Lieshout <[email protected]> * Added logic to remove rows with trailing nan for a feature. Signed-off-by: Marnix van Lieshout <[email protected]> * Add tests. Signed-off-by: Marnix van Lieshout <[email protected]> * Restore pyproject.toml and uv.lock from origin/feature/626-feature-engineering Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting, formatting Signed-off-by: Marnix van Lieshout <[email protected]> * More linting and formatting Signed-off-by: Marnix van Lieshout <[email protected]> * Fix typing Signed-off-by: Marnix van Lieshout <[email protected]> * Review comments Signed-off-by: Marnix van Lieshout <[email protected]> * Pyright Signed-off-by: Marnix van Lieshout <[email protected]> * Changes based on review comments Signed-off-by: Marnix van Lieshout <[email protected]> * Fix tests and warnings Signed-off-by: Marnix van Lieshout <[email protected]> * Fix warnings Signed-off-by: Marnix van Lieshout <[email protected]> * Fix fixtures. Signed-off-by: Marnix van Lieshout <[email protected]> * Clean up Signed-off-by: Marnix van Lieshout <[email protected]> * Add warning assert to test Signed-off-by: Marnix van Lieshout <[email protected]> * Fix attribute type Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting Signed-off-by: Marnix van Lieshout <[email protected]> * Reset uv.lock Signed-off-by: Marnix van Lieshout <[email protected]> * Config as part of init Signed-off-by: Marnix van Lieshout <[email protected]> * Remove type ignores Signed-off-by: Marnix van Lieshout <[email protected]> * Extend transform from baseconfig Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting Signed-off-by: Marnix van Lieshout <[email protected]> * Validate for fill value when constant strategy. Signed-off-by: Marnix van Lieshout <[email protected]> * Use default value for missing_value Signed-off-by: Marnix van Lieshout <[email protected]> * Move imputer to private attribute. Signed-off-by: Marnix van Lieshout <[email protected]> * Fix warning Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]> * initial commit weather transforms Signed-off-by: lschilders <[email protected]> * fix type errors * Add flatliner check transform Signed-off-by: Marnix van Lieshout <[email protected]> * fix most type and formatting errors * fix all errors except for weather transforms * fix radiation_derived_features tests * Remove redundant comment Signed-off-by: Marnix van Lieshout <[email protected]> * Fix import statement Signed-off-by: Marnix van Lieshout <[email protected]> * vectorize feature clipper computation * rename scaler to scaler_transform * rename rolling aggregate features * formatting Signed-off-by: lschilders <[email protected]> * Add absolute tolerance arg. Update docstrings and tests. Signed-off-by: Marnix van Lieshout <[email protected]> * feature(#628): Update feature naming. Signed-off-by: Egor Dmitriev <[email protected]> * Improve docstring Signed-off-by: Marnix van Lieshout <[email protected]> * feature(#628): Updated forecasting transforms. Signed-off-by: Egor Dmitriev <[email protected]> * Added flatliner detection to transform Signed-off-by: Marnix van Lieshout <[email protected]> * Add option to give custom load column name Signed-off-by: Marnix van Lieshout <[email protected]> * Fix order of exceptions Signed-off-by: Marnix van Lieshout <[email protected]> * Add overrides Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Frank Kreuwel <[email protected]> Signed-off-by: Bart Pleiter <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: lschilders <[email protected]> Signed-off-by: Marnix van Lieshout <[email protected]> Co-authored-by: Frank Kreuwel <[email protected]> Co-authored-by: Bart Pleiter <[email protected]> Co-authored-by: Egor Dmitriev <[email protected]> Co-authored-by: lschilders <[email protected]> Co-authored-by: Copilot <[email protected]>
* feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Updated package deps. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(#628): Added feature transform pipeline. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Improved doc example. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-models/tests/unit/feature_engineering/validation_transforms/test_flatliner_check.py Co-authored-by: Marnix van Lieshout <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Fixed doctest for FeaturPipeline. Added repr to leadtime. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Improved test cases for feature pipeline. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Improved test cases for horizon splitting. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Marnix van Lieshout <[email protected]>
…t part can have duplicate timestamps in index. (#665) * feature(#628): Added feature transform pipeline. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Improved doc example. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Added lag transform. Fixed bug where timeseries dataset part can have duplicate timestamps in index. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#628): Clarified test doc. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* Add file Signed-off-by: Marnix van Lieshout <[email protected]> * Add calculations with references Signed-off-by: Marnix van Lieshout <[email protected]> * Fix quality checks Signed-off-by: Marnix van Lieshout <[email protected]> * Make stateless, fix doctest Signed-off-by: Marnix van Lieshout <[email protected]> * Rename to atmosphere derived features Signed-off-by: Marnix van Lieshout <[email protected]> * Rename test file Signed-off-by: Marnix van Lieshout <[email protected]> * Add hardcoded values Signed-off-by: Marnix van Lieshout <[email protected]> * Add more info and DOI Signed-off-by: Marnix van Lieshout <[email protected]> * Remove redundant comment Signed-off-by: Marnix van Lieshout <[email protected]> * Add more info to docstring Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]>
* Init commit Signed-off-by: Marnix van Lieshout <[email protected]> * Clean up Signed-off-by: Marnix van Lieshout <[email protected]> * Improve docstrings Signed-off-by: Marnix van Lieshout <[email protected]> * Improve naming Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]>
…by fixing github links to docs and disabling source button. (#667) * docs(#656): Updated code style guide regarding types. Signed-off-by: Egor Dmitriev <[email protected]> * docs(#661): Fixed links to edit files on github. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(#627): Initial model interface implementation. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#627): Documented model interface mixins and updated documentation guidelines with new Invariant section. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#627): Added tests for combine_horizon_forecasts. Updated docstrings / and tests for mixins. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2391): Added tests for constant median forecaster and documentation to validated datasets. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#627): Updated code style guide. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2391): Added docs for forecaster implementations. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2391): Added docs for forecaster implementations. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2391): Fixed tests so fitting is more adaptable for multi horizon adapter. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#627): Minor fixes after feedback. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
…torage, callbacks. Refactor by reorganizing transforms. (#674) * refactor(#633): Moved feature engineering into transforms. Favoring a more flat structure for different operations on datasets. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added storage, callback and workflow interfaces. Added an example and implemented example storage. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added tests for new components. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added docstings to the new pipelines. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added docstings to the storage. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Renamed postprocessing pipeline to ForecastTransformPipeline. Refactored holiday features transform to avoid patching since it often breaks tests due to minor changes. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-models/src/openstef_models/models/forecasting_model.py Co-authored-by: Lars Schilders <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Updated dependency groups. Implemented is_fitted. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Lars Schilders <[email protected]>
* add rIQD and relative pinball loss metrics Signed-off-by: MentReeze <[email protected]> * docstring fix Co-authored-by: Lars Schilders <[email protected]> Signed-off-by: MentReeze <[email protected]> * add unit test for symmetric quantile logic of rIQD provider Signed-off-by: MentReeze <[email protected]> --------- Signed-off-by: MentReeze <[email protected]> Co-authored-by: Lars Schilders <[email protected]>
…n data (#676) * Add the option to plot timeseries without showing interpolated lines where gaps in data Signed-off-by: MentReeze <[email protected]> * Fix ruff formatting issues Signed-off-by: MentReeze <[email protected]> --------- Signed-off-by: MentReeze <[email protected]>
…ded split component pipeline. (#677) * feature(#633): Moved model mixins into a single namespace inside models. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Updated transform interfaces. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Refactored feature pipeline and transform base. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added component splitting. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Refactored transforms, predictors, and statefulness. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Improved doctests and docstrings. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Improved docstrings. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added tests voor multi horizon transform adapter. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Added support for validation set during training. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#633): Changed dataset to data naming for consistency in forecasting model. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* feature(#687): Implemented xgboost model. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Added tests for loss objectives. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Added docstrings for xgboost implementation. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Improved tests. Signed-off-by: Egor Dmitriev <[email protected]> * Update packages/openstef-models/src/openstef_models/models/forecasting/xgboost_forecaster.py Co-authored-by: Lars Schilders <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Documented model state. Moved input_data filtering into forecast input dataset. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Lars Schilders <[email protected]>
* feature: add sample_weight_column to ForecastInputDataset * add doctest example * add code and config for gblinear * feature(#687): Implemented xgboost model. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Added tests for loss objectives. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Added docstrings for xgboost implementation. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Improved tests. Signed-off-by: Egor Dmitriev <[email protected]> * refactor gblinear and tests * Update packages/openstef-models/src/openstef_models/models/forecasting/xgboost_forecaster.py Co-authored-by: Lars Schilders <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * feature(#687): Documented model state. Moved input_data filtering into forecast input dataset. Signed-off-by: Egor Dmitriev <[email protected]> * align GBLinear implementation with XGBoost and add booster type checking * add GBLinear implementation * revert changes to types * revert changes to pyproject * revert changes to pyproject * add doctest SKIP --------- Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Egor Dmitriev <[email protected]>
…ions (#813) When rolling_aggregate_features is empty (e.g., ato_regions, grid_losses), the RollingAggregatesAdder transform is no longer added to the pipeline. Previously it was always added, and fit() would crash with ValueError: No objects to concatenate when calling pandas rolling().agg([]). Signed-off-by: Egor Dmitriev <[email protected]>
The bare 'mlflow' pattern matched anywhere in the tree, which caused hatchling to exclude packages/openstef-models/src/openstef_models/integrations/mlflow/ from the built wheel. Root-anchoring with '/mlflow' limits the match to only the top-level mlflow directory (local MLflow data). Broken since v4.0.0.a17 (commit 1bcf71d). Signed-off-by: Egor Dmitriev <[email protected]>
* Added Lightgbm, LightGBM Linear Trees and Hybrid Stacking Forecasters * Fixed small issues * Ruff compliance * fixed quality checks * Fixed last issues, Signed-off-by: Lars van Someren <[email protected]> * fixed comments * Refactor LightGBM to LGBM * Update LGBM and LGBMLinear defaults, fixed comments * Fixed comments * Added SkopsModelSerializer * Fixed issues * Gitignore optimization and dev sandbox * Added MultiQuantileAdapter Class * small fix * Hybrid V2 * Small fix * Squashed commit of the following: commit 37089b8 Author: Egor Dmitriev <[email protected]> Date: Mon Nov 17 15:29:59 2025 +0100 fix(#728): Fixed parallelism stability issues, and gblinear feature pipeline. (#752) * fix(STEF-2475): Added loky as default option for parallelism since fork causes instabilities for xgboost results. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2475): Added better support for flatliners and predicting when data is sparse. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2475): Feature handing improvements for gblinear. Like imputation, nan dropping, and checking if features are available. Signed-off-by: Egor Dmitriev <[email protected]> * fix(#728): Added checks on metrics to gracefully handle empty data. Added flatline filtering during evalution. Signed-off-by: Egor Dmitriev <[email protected]> * fix(#728): Updated xgboost to skip scaling on empty prediction. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2475): Added parallelism parameters. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> commit a85a3f7 Author: Egor Dmitriev <[email protected]> Date: Fri Nov 14 14:31:34 2025 +0100 fix(STEF-2475): Fixed rolling aggregate adder by adding forward filling and stating support for only one horizon. (#750) Signed-off-by: Egor Dmitriev <[email protected]> commit 4f0c664 Author: Egor Dmitriev <[email protected]> Date: Thu Nov 13 16:54:15 2025 +0100 feature: Disabled data cutoff by default to be consistent with openstef 3. And other minor improvements. (#748) commit 493126e Author: Egor Dmitriev <[email protected]> Date: Thu Nov 13 16:12:35 2025 +0100 fix(STEF-2475) fix and refactor backtesting iction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits. (#747) * fix: Fixed data collation during backtesting. Renamed horizon to prediction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits. * fix: Formatting. Signed-off-by: Egor Dmitriev <[email protected]> * fix: Formatting. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> commit 6b1da44 Author: Egor Dmitriev <[email protected]> Date: Thu Nov 13 16:05:32 2025 +0100 feature: forecaster hyperparams and eval metrics (#746) * feature(#729) Removed to_state and from_state methods in favor of builtin python state saving functions. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Fixed issue where generic transform pipeline could not be serialized. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Added more state saving tests Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Added more state saving tests Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Added more state saving tests Signed-off-by: Egor Dmitriev <[email protected]> * feature: standardized objective function. Added custom evaluation functions for forecasters. * fix: Formatting. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> * set silence * small fix * Fix final learner * fixed lgbm efficiency * updated lgbm linear params * Fixed type and quality issues * First Version Sample Weighting Approach Signed-off-by: Lars van Someren <[email protected]> * MetaForecasterClass Signed-off-by: Lars van Someren <[email protected]> * Research/v4.1.0 additional forecasters (#765) * Added Lightgbm, LightGBM Linear Trees and Hybrid Stacking Forecasters * Fixed small issues * Ruff compliance * fixed quality checks * Fixed last issues, Signed-off-by: Lars van Someren <[email protected]> * fixed comments * Refactor LightGBM to LGBM * Update LGBM and LGBMLinear defaults, fixed comments * Fixed comments * Added SkopsModelSerializer * Fixed issues * Gitignore optimization and dev sandbox * Added MultiQuantileAdapter Class * small fix * Hybrid V2 * Small fix * Squashed commit of the following: commit 37089b8 Author: Egor Dmitriev <[email protected]> Date: Mon Nov 17 15:29:59 2025 +0100 fix(#728): Fixed parallelism stability issues, and gblinear feature pipeline. (#752) * fix(STEF-2475): Added loky as default option for parallelism since fork causes instabilities for xgboost results. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2475): Added better support for flatliners and predicting when data is sparse. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2475): Feature handing improvements for gblinear. Like imputation, nan dropping, and checking if features are available. Signed-off-by: Egor Dmitriev <[email protected]> * fix(#728): Added checks on metrics to gracefully handle empty data. Added flatline filtering during evalution. Signed-off-by: Egor Dmitriev <[email protected]> * fix(#728): Updated xgboost to skip scaling on empty prediction. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2475): Added parallelism parameters. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> commit a85a3f7 Author: Egor Dmitriev <[email protected]> Date: Fri Nov 14 14:31:34 2025 +0100 fix(STEF-2475): Fixed rolling aggregate adder by adding forward filling and stating support for only one horizon. (#750) Signed-off-by: Egor Dmitriev <[email protected]> commit 4f0c664 Author: Egor Dmitriev <[email protected]> Date: Thu Nov 13 16:54:15 2025 +0100 feature: Disabled data cutoff by default to be consistent with openstef 3. And other minor improvements. (#748) commit 493126e Author: Egor Dmitriev <[email protected]> Date: Thu Nov 13 16:12:35 2025 +0100 fix(STEF-2475) fix and refactor backtesting iction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits. (#747) * fix: Fixed data collation during backtesting. Renamed horizon to prediction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits. * fix: Formatting. Signed-off-by: Egor Dmitriev <[email protected]> * fix: Formatting. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> commit 6b1da44 Author: Egor Dmitriev <[email protected]> Date: Thu Nov 13 16:05:32 2025 +0100 feature: forecaster hyperparams and eval metrics (#746) * feature(#729) Removed to_state and from_state methods in favor of builtin python state saving functions. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Fixed issue where generic transform pipeline could not be serialized. Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Added more state saving tests Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Added more state saving tests Signed-off-by: Egor Dmitriev <[email protected]> * feature(#729): Added more state saving tests Signed-off-by: Egor Dmitriev <[email protected]> * feature: standardized objective function. Added custom evaluation functions for forecasters. * fix: Formatting. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> * set silence * small fix * Fix final learner * fixed lgbm efficiency * updated lgbm linear params * Fixed type and quality issues * remove depricated files Signed-off-by: Lars van Someren <[email protected]> * change: Fixed dependencies to align more with the current release. Signed-off-by: Egor Dmitriev <[email protected]> * change: Style fixes. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Lars van Someren <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Co-authored-by: Egor Dmitriev <[email protected]> * fix merge issue Signed-off-by: Lars van Someren <[email protected]> * Fixed type Issues Signed-off-by: Lars van Someren <[email protected]> * Introduced openstef_metalearning Signed-off-by: Lars van Someren <[email protected]> * ResidualForecaster + refactoring Signed-off-by: Lars van Someren <[email protected]> * Testing and fixes on Learned Weights Forecaster Signed-off-by: Lars van Someren <[email protected]> * FinalLearner PreProcessor Signed-off-by: Lars van Someren <[email protected]> * Fixed benchmark references Signed-off-by: Lars van Someren <[email protected]> * Added additional Feature logic to StackingForecaster Signed-off-by: Lars van Someren <[email protected]> * added example to openstef Meta Signed-off-by: Lars van Someren <[email protected]> * RulesForecaster with dummy features Signed-off-by: Lars van Someren <[email protected]> * Updated feature specification Signed-off-by: Lars van Someren <[email protected]> * entered flagger feature in new architecture * Fix sample weights Signed-off-by: Lars van Someren <[email protected]> * Fixes Signed-off-by: Lars van Someren <[email protected]> * PR compliant Signed-off-by: Lars van Someren <[email protected]> * Ensemble Forecast Dataset Signed-off-by: Lars van Someren <[email protected]> * Make PR compliant Signed-off-by: Lars van Someren <[email protected]> * fixed toml Signed-off-by: Lars van Someren <[email protected]> * Really fixed the TOML Signed-off-by: Lars van Someren <[email protected]> * Renamed FinalLearner to Forecast Combiner. Eliminated redundant classes Signed-off-by: Lars van Someren <[email protected]> * fixed small issues Signed-off-by: Lars van Someren <[email protected]> * Major Refactor, Working Version Signed-off-by: Lars van Someren <[email protected]> * Fixed tests Signed-off-by: Lars van Someren <[email protected]> * Prepared TODOs for Florian Signed-off-by: Lars van Someren <[email protected]> * Small fix Signed-off-by: Lars van Someren <[email protected]> * Made PR Compliant Signed-off-by: Lars van Someren <[email protected]> * BugFix Signed-off-by: Lars van Someren <[email protected]> * fixes Signed-off-by: Lars van Someren <[email protected]> * bug fixes Signed-off-by: Lars van Someren <[email protected]> * added learned weights contributions * Added Feature Contributions Residual Forecaster and Stacking Forecaster can now predict model contributions. Regular forecasters (EXCEPT LGBM Linear) can predict feature contributions * Bugfixes Signed-off-by: Lars van Someren <[email protected]> * fixes Signed-off-by: Lars van Someren <[email protected]> * Squashed commit of the following: commit 6f88d72 Author: Lars van Someren <[email protected]> Date: Mon Dec 8 09:46:57 2025 +0100 Bugfixes Signed-off-by: Lars van Someren <[email protected]> commit b44fd92 Author: Lars van Someren <[email protected]> Date: Thu Dec 4 14:39:31 2025 +0100 bug fixes Signed-off-by: Lars van Someren <[email protected]> commit e212448 Author: Lars van Someren <[email protected]> Date: Thu Dec 4 12:38:24 2025 +0100 fixes Signed-off-by: Lars van Someren <[email protected]> commit eb775e4 Author: Lars van Someren <[email protected]> Date: Thu Dec 4 11:40:44 2025 +0100 BugFix Signed-off-by: Lars van Someren <[email protected]> commit c33ce93 Author: Lars van Someren <[email protected]> Date: Wed Dec 3 14:15:06 2025 +0100 Made PR Compliant Signed-off-by: Lars van Someren <[email protected]> * Fixes Signed-off-by: Lars van Someren <[email protected]> * fixed tests Signed-off-by: Lars van Someren <[email protected]> * small fix Signed-off-by: Lars van Someren <[email protected]> * Stacking Bugfix Signed-off-by: Lars van Someren <[email protected]> * Added hard Forecast Selection Signed-off-by: Lars van Someren <[email protected]> * Improved data handling in EnsembleForecasting model, correct data splitting and Model Fit Result. Validation and test data can now be fully used Signed-off-by: Lars van Someren <[email protected]> * Migrated Flagger and Selector to OpenSTEF Models transforms Signed-off-by: Lars van Someren <[email protected]> * Fixed restore target Forecast Combiner Signed-off-by: Lars van Someren <[email protected]> * Streamline logging statements, Fix quality Signed-off-by: Lars van Someren <[email protected]> * Resolved comments, fixed bug Signed-off-by: Lars van Someren <[email protected]> * Moved example Signed-off-by: Lars van Someren <[email protected]> * Bring 4.1 up to date with release Squashed commit of the following: commit 6d140bc Author: Lars Schilders <[email protected]> Date: Wed Dec 17 10:33:19 2025 +0100 feature: add regex pattern matching in FeatureSelection and fix combine bug (#787) commit 32a42bb Author: Lars Schilders <[email protected]> Date: Tue Dec 16 13:50:40 2025 +0100 feature: Selector transform (#786) * feature: add Selector transform * add ForecastInputDataset testcases * add selected_features to presets * add doctest commit d3977b1 Author: Egor Dmitriev <[email protected]> Date: Mon Dec 15 09:17:38 2025 +0100 feature: added tutorials for basic functionality. Added convenience method for simple openstef baselines. (#785) * feature: Added tutorial start. Signed-off-by: Egor Dmitriev <[email protected]> * feature: Added example notebooks. First draft. Signed-off-by: Egor Dmitriev <[email protected]> * chore(examples): add examples workspace project and register it in workspace; update lock * fix(lint): add missing docstrings in baselines package (D104, D103) * chore(examples): add examples workspace project and register it in workspace; update lock * chore(examples): Updated text in examples. --------- Signed-off-by: Egor Dmitriev <[email protected]> commit 8a4097c Author: Bart Pleiter <[email protected]> Date: Wed Dec 10 16:19:48 2025 +0100 fix: exclude stdev column from quantile column checking. (#783) * fix: exclude stdev column from quantile column checking. Signed-off-by: Bart Pleiter <[email protected]> * fix: duplicate removed. Signed-off-by: Bart Pleiter <[email protected]> * fix: type Signed-off-by: Bart Pleiter <[email protected]> --------- Signed-off-by: Bart Pleiter <[email protected]> commit 43987fc Author: Egor Dmitriev <[email protected]> Date: Wed Dec 10 10:24:32 2025 +0100 fix(STEF-2549): Added none check for model end date from mlflow. Added experiment tags. (#782) Signed-off-by: Egor Dmitriev <[email protected]> commit 1891009 Author: Lars Schilders <[email protected]> Date: Tue Dec 9 14:40:40 2025 +0100 feature: check for model config change and skip model selection (#781) * feature: check for model config change and skip model selection * changed checking model compatibility * check for tag compatibility only * fix tests * rename new methods in callback commit c37ac92 Author: Lars Schilders <[email protected]> Date: Tue Dec 9 09:22:58 2025 +0100 fix: clip values of wind and solar components to below 0 (#779) * fix: clip values of wind and solar components to below 0 * add test for not all components zero commit 3eb7e69 Author: Egor Dmitriev <[email protected]> Date: Mon Dec 8 15:55:16 2025 +0100 feat(mlflow): suppress MLflow emoji URL logs (#780) * feat(mlflow): suppress MLflow emoji URL logs Add MLFLOW_SUPPRESS_PRINTING_URL_TO_STDOUT=true environment variable to prevent MLflow from printing 'View run...' messages with emojis that don't comply with ECS JSON logging format. Signed-off-by: Egor Dmitriev <[email protected]> * feature: Style fixes. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> commit eca628e Author: Lars Schilders <[email protected]> Date: Fri Dec 5 16:27:53 2025 +0100 feature: nonzero flatliner preset (#777) * add predict_nonzero_flatliner to presets * remove redundant validate_required_columns commit 61e1699 Author: Lars Schilders <[email protected]> Date: Fri Dec 5 15:37:05 2025 +0100 feature: add standard devation column to ForecastDataset and add it in ConfidenceIntervalApplicator (#778) * feature: add standard devation column to ForecastDataset and add it in ConfidenceIntervalApplicator * simplify code for adding column commit 4f70d00 Author: Lars Schilders <[email protected]> Date: Fri Dec 5 10:41:23 2025 +0100 chore: change radiation unit to Wm-2 (#776) * chore: change expected radiation unit to W/m-2 * change values in test for radiation features adder * fix docs for dni/gti unit * formatting commit 71ac428 Author: Bart Pleiter <[email protected]> Date: Wed Dec 3 09:45:34 2025 +0100 feature: added use_median option to flatliner forecaster so it predic… (#773) * feature: added use_median option to flatliner forecaster so it predicts the median of the training data. Signed-off-by: Bart Pleiter <[email protected]> * feature: improved naming to predict_median. Signed-off-by: Bart Pleiter <[email protected]> --------- Signed-off-by: Bart Pleiter <[email protected]> commit 45ca37f Author: Lars Schilders <[email protected]> Date: Wed Nov 26 15:45:07 2025 +0100 fix: fixes in EvaluationPipeline and TimeSeriesPlotter (#769) * Remove target column from predictions to avoid duplication for lead_times * get sample_interval class attr commit ee41442 Author: Egor Dmitriev <[email protected]> Date: Wed Nov 26 14:01:16 2025 +0100 fix: Improved mlflow to use run names and load proper models for reuse. Fixed time series plotter to use correct sample interval paramter. (#768) * feature: Improved mlflow to use run names and load proper models for reuse. Fixed time series plotter to use correct sample interval paramter. Signed-off-by: Egor Dmitriev <[email protected]> * feature(STEF-2551): Fixed path. Changed run_name to step_name in backtester. Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]> commit 7deb69e Author: Bart Pleiter <[email protected]> Date: Fri Nov 21 14:42:54 2025 +0100 chore: replaced alliander emails with lfenergy email. (#767) Signed-off-by: Bart Pleiter <[email protected]> Signed-off-by: Lars van Someren <[email protected]> * Integrated changes to beam structure Signed-off-by: Lars van Someren <[email protected]> * make PR compliant Signed-off-by: Lars van Someren <[email protected]> * Fix when aggregation functions empty. Signed-off-by: Marnix van Lieshout <[email protected]> * Improve naming Signed-off-by: Marnix van Lieshout <[email protected]> * Cleaning up Signed-off-by: Marnix van Lieshout <[email protected]> * Change how config's are made and used for combiner and stacking models. Signed-off-by: Marnix van Lieshout <[email protected]> * Cleaning up Signed-off-by: Marnix van Lieshout <[email protected]> * Move skops code to separate branch Signed-off-by: Marnix van Lieshout <[email protected]> * Move rules combiner code to separate branch Signed-off-by: Marnix van Lieshout <[email protected]> * Cleaning up Signed-off-by: Marnix van Lieshout <[email protected]> * Cleaning up Signed-off-by: Marnix van Lieshout <[email protected]> * Improve hyperparam naming Signed-off-by: Marnix van Lieshout <[email protected]> * Remove openstef-meta dependency in openstef-models Signed-off-by: Marnix van Lieshout <[email protected]> * Separating openstef-models and openstef-meta WIP Signed-off-by: Marnix van Lieshout <[email protected]> * Move residual model to separate branch Signed-off-by: Marnix van Lieshout <[email protected]> * Rename regression tests to integration Signed-off-by: Marnix van Lieshout <[email protected]> * Move forecaster back for now Signed-off-by: Marnix van Lieshout <[email protected]> * Use sample weight config in models Signed-off-by: Marnix van Lieshout <[email protected]> * Add importances to median forecaster Signed-off-by: Marnix van Lieshout <[email protected]> * Rename for consistency Signed-off-by: Marnix van Lieshout <[email protected]> * Move base mlflow callback to separate file Signed-off-by: Marnix van Lieshout <[email protected]> * Clean up docstring Signed-off-by: Marnix van Lieshout <[email protected]> * Add explainability to combiner models Signed-off-by: Marnix van Lieshout <[email protected]> * Make combiners explainable Signed-off-by: Marnix van Lieshout <[email protected]> * Delete empty folder Signed-off-by: Marnix van Lieshout <[email protected]> * Use protocols Signed-off-by: Marnix van Lieshout <[email protected]> * Rename test file Signed-off-by: Marnix van Lieshout <[email protected]> * Reset pinball_losses changes Signed-off-by: Marnix van Lieshout <[email protected]> * Fix linting issues Signed-off-by: Marnix van Lieshout <[email protected]> * Add base models to tags MLflow Signed-off-by: Marnix van Lieshout <[email protected]> * Improve what is stored in MLFlow Signed-off-by: Marnix van Lieshout <[email protected]> * Formatting Signed-off-by: Marnix van Lieshout <[email protected]> * refactor(STEF-2702): Meta rework v2 — flatten configs, abstract hparams, clean up presets (#818) * fix(STEF-2802): skip RollingAggregatesAdder when no aggregation functions (#813) When rolling_aggregate_features is empty (e.g., ato_regions, grid_losses), the RollingAggregatesAdder transform is no longer added to the pipeline. Previously it was always added, and fit() would crash with ValueError: No objects to concatenate when calling pandas rolling().agg([]). Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2802): anchor mlflow gitignore patterns to repo root (#814) The bare 'mlflow' pattern matched anywhere in the tree, which caused hatchling to exclude packages/openstef-models/src/openstef_models/integrations/mlflow/ from the built wheel. Root-anchoring with '/mlflow' limits the match to only the top-level mlflow directory (local MLflow data). Broken since v4.0.0.a17 (commit 1bcf71d). Signed-off-by: Egor Dmitriev <[email protected]> * refactor(STEF-2702): extract BaseForecastingModel and ContributionsMixin - Extract BaseForecastingModel as abstract base with hyperparams property - Create ContributionsMixin with predict_contributions -> TimeSeriesDataset - Remove predict_contributions from ExplainableForecaster (now ContributionsMixin) - Implement ContributionsMixin on all ML forecasters (XGBoost, GBLinear, LGBM, LGBMLinear) - Implement ContributionsMixin on baseline forecasters (flatliner, median, constant_median, base_case) - Defer lightgbm imports to __init__() with MissingExtraError guard - Inline _get_primary_hyperparams into model.hyperparams property - Update mlflow callback and custom workflow to use BaseForecastingModel - Update tests for new predict_contributions signature * refactor(STEF-2702): flatten combiners, rearchitect StackingCombiner, prune low-value tests - ForecastCombiner IS its config (BaseConfig + Predictor + ABC) - Remove ForecastCombinerConfig entirely - WeightsCombiner/StackingCombiner: flatten fields, use PrivateAttr for mutable state - StackingCombiner: accept meta_forecaster template, clone via config.model_copy() - Remove factory chain (forecaster_class -> Config -> forecaster_from_config) - Add hyperparams field to ForecastCombiner base (no more getattr reflection) - Extract _prepare_input helper, simplify predict/contributions/importances - Delete test_forecast_combiner.py (6 tests testing pydantic/stdlib only) - Remove test_initialization from combiner tests (covered by fit_predict) - Remove test_init_uses_defaults (tested pydantic defaults) - All 469 tests pass across core+models+meta * refactor(STEF-2702): flatten forecasters into BaseConfig, add abstract hparams property * refactor(STEF-2702): column separator, lightgbm optional, move classification to combiner - D5: Move get_best_forecaster_labels/pinball classification from EnsembleForecastDataset to WeightsCombiner._classify_best_forecaster - D6: Change ensemble column separator from '_' to '__' (ENSEMBLE_COLUMN_SEP), add forecaster name validation, remove dead get_quantile_feature_name - Phase 1.5: Make lightgbm optional dependency (optional-dependencies.lgbm), add MissingExtraError guards to combiner get_classifier methods - Phase 3.2: Replace isinstance(model, EnsembleForecastingModel) with polymorphic try/except NotImplementedError in beam backtest * refactor(STEF-2702): abstract hparams on ForecastCombiner, fix all lint/type errors, add inline docs * refactor(STEF-2702): rename EnsembleWorkflowConfig to EnsembleForecastingWorkflowConfig * refactor(STEF-2702): clean up forecasting_workflow preset, extract helpers, prefix internals * fix(STEF-2702): address PR review — remove D1 reference, rename fixture * feat(STEF-2702): validate horizons consistency at EnsembleForecastingModel construction * fix(STEF-2702): regenerate lockfile against PyPI (remove JFrog URLs) --------- Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Lars van Someren <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Marnix van Lieshout <[email protected]> Co-authored-by: Egor Dmitriev <[email protected]> Co-authored-by: floriangoethals <[email protected]> Co-authored-by: Marnix van Lieshout <[email protected]>
Signed-off-by: Marnix van Lieshout <[email protected]>
* refactor(STEF-2702): push polymorphism into models, eliminate isinstance protocols - SubsetMetric.to_flat_dict() for metrics self-serialization - ModelFitResult.metrics_to_flat_dict() with component_fit_results override - BaseForecastingModel: component_hyperparams, get_explainable_components() - ForecastCombiner extends ExplainableForecaster (gains plot_feature_importances) - EnsembleForecastingModel/EnsembleModelFitResult override all polymorphic methods - MLflow callback: delete 3 protocols, 5 isinstance checks -> polymorphic calls - Fix penalty bug: lower_is_better now multiplies (was dividing) * refactor(STEF-2702): extract normalize_to_unit_sum, dedup feature importances - Add normalize_to_unit_sum() pipe-compatible utility in openstef_core.utils.pandas - Replace duplicated normalization logic in 4 forecasters (xgboost, gblinear, lgbm, lgbmlinear) - All now use weights_df.pipe(normalize_to_unit_sum) * fix: make AvailableAt.__str__() Windows-safe by removing colon Change DnTHH:MM format to DnTHHMM (e.g. D-1T0600 instead of D-1T06:00). Colons are illegal in Windows file paths, breaking benchmark output directories. from_string() now accepts both formats for backward compatibility. * feat(STEF-2702): make openstef-models/meta optional for beam baselines * refactor(STEF-2702): replace workflow factory with template + model_copy - Replace workflow_factory Callable with workflow_template field - Add with_run_name() method for type-safe deep copy - Add kind discriminator to ForecastingWorkflowConfig (single) and EnsembleForecastingWorkflowConfig (ensemble) - Simplify factory: use kind-based narrowing, raise MissingExtraError - Delete WorkflowCreationContext (no longer needed) - Remove both pyright: ignore[reportArgumentType] suppressions * test(STEF-2702): add template pattern smoke tests for OpenSTEF4BacktestForecaster - test_fit_does_not_mutate_template: verifies template immutability after fit - test_fit_then_predict_returns_forecast: e2e smoke test for fit→predict path * chore(STEF-2702): add openstef-meta to release pipeline - Add missing [build-system] section to openstef-meta pyproject.toml - Add openstef-meta to poe version task (version bump + root pinning) - Fix baselines extra version range: >=4.0.0.dev0,<5 (was >=0.0.1,<1) - Add openstef-meta to root [all] optional extra - Add openstef-meta to licensecheck ignore_packages * refactor(STEF-2702): extend test utility, address review comments - Add include_atmosphere/price/available_at options to create_synthetic_forecasting_dataset for realistic test data - Simplify test fixture to use the shared utility - Remove verbose parenthetical comments per review feedback
…ty (#826) - Add normalize_tracking_uri() helper that converts local file paths to file:/// URIs, handling relative paths, absolute POSIX paths, and Windows drive-letter paths (e.g. D:\mlflow) - Apply normalization in MLFlowStorage.model_post_init() so all users get correct URIs regardless of how they construct tracking_uri - Add parametrized tests covering 8 URI variants including a mock-based Windows drive letter test
Signed-off-by: lschilders <[email protected]>
* Add run_name. Set default weights. Add confidence interval applicator postprocessing. Signed-off-by: Marnix van Lieshout <[email protected]> * Add lgbm extra for openstef-models dep in openstef-meta Signed-off-by: Marnix van Lieshout <[email protected]> * No sample weights standard for lgbm Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]>
* Add run_name. Set default weights. Add confidence interval applicator postprocessing. Signed-off-by: Marnix van Lieshout <[email protected]> * Add lgbm extra for openstef-models dep in openstef-meta Signed-off-by: Marnix van Lieshout <[email protected]> * No sample weights standard for lgbm Signed-off-by: Marnix van Lieshout <[email protected]> * transform -> fit_transform Signed-off-by: Marnix van Lieshout <[email protected]> * Split postprocessing into common, per-forecaster and combiner Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]>
* feature: add apply() to AvailableAt and tz-aware option * feature: refactor AvailableAt to use day_offset and time_of_day rather than lag_from_day * feature: (de)serialization support for tz-aware AvailableAt * feature: improved docstring * feature: add tests for apply_index Signed-off-by: lschilders <[email protected]> --------- Signed-off-by: lschilders <[email protected]>
Signed-off-by: Marnix van Lieshout <[email protected]>
…833) * fix: timezone handling for offset in AvailableAt apply() (matching apply_index()) * feature: add new metric NMAE (normalized MAE) * add MAEProvider * make allow_nan configurable * refactored NMAE into rMAE with norm_value Signed-off-by: lschilders <[email protected]> * formatting Signed-off-by: lschilders <[email protected]> * add allow_nan to rMAE --------- Signed-off-by: lschilders <[email protected]>
* Update workflows Signed-off-by: Marnix van Lieshout <[email protected]> * Revert doc change Signed-off-by: Marnix van Lieshout <[email protected]> --------- Signed-off-by: Marnix van Lieshout <[email protected]>
* fix(STEF-2854): handle InsufficientlyCompleteError during backtest training OpenSTEF4BacktestForecaster.fit() now catches InsufficientlyCompleteError alongside FlatlinerDetectedError. When a training window has insufficient non-NaN data, the training event is skipped and the previous model is retained instead of crashing the entire target backtest. Signed-off-by: Egor Dmitriev <[email protected]> * test(STEF-2854): replace mock with real NaN data in insufficient-data test Use all-NaN load data with model_reuse_enable=False to trigger InsufficientlyCompleteError naturally instead of patching workflow.fit. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): return None from predict() when no model fitted When the first fit fails due to InsufficientlyCompleteError, _workflow stays None. predict() now returns None (like flatliner) instead of raising NotFittedError, letting the benchmark pipeline skip gracefully. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): make WindowedMetricVisualization robust to missing data Skip runs/targets with no windowed metrics instead of raising ValueError. Returns an HTML placeholder when all items in a visualization are empty. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): fix combiner label/weight shape mismatch and Quantile serialization Two fixes: 1. learned_weights_combiner.py: Filter labels to match combined_data index after inner join drops rows from additional_features. Fixes ValueError: operands could not be broadcast together. 2. types.py: Add Pydantic serializer to Quantile to suppress PydanticSerializationUnexpectedValue warnings. Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): add Pydantic serializer to Quantile to suppress warnings Quantile.__get_pydantic_core_schema__ only defined a validator but no serializer. When Quantile values appear as dict keys in a union type (e.g., QuantileOrGlobal = Quantile | Literal['global']), Pydantic emits PydanticSerializationUnexpectedValue warnings during model_dump_json(). Add a plain_serializer_function_ser_schema(float) so Pydantic knows how to serialize Quantile as a float, preventing the warning. Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): raise InsufficientlyCompleteError for empty datasets in train/test split chronological_train_test_split crashed with IndexError when the dataset had fewer than 2 unique timestamps. This happens during ensemble backtest when a base forecaster's preprocessed data is empty. Now raises InsufficientlyCompleteError which is caught by the backtest harness. Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): raise InsufficientlyCompleteError on empty combiner data after inner join Signed-off-by: Egor Dmitriev <[email protected]> * feat(STEF-2854): add strict parameter to BenchmarkComparisonPipeline.run() Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): use 'global' subdirectory for RUN_AND_GROUP analysis scope The RUN_AND_GROUP scope was saving directly to the base analysis dir, making has_analysis_output fail to locate it and colliding with group-level outputs. Store in a 'global' subdirectory to match the group-level pattern (group_name/global). Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): renormalize ensemble weights when base model predictions are NaN When a base model cannot predict certain timestamps (e.g. gblinear limited to 2-day weather horizon while lgbm predicts 7 days), the combiner must redistribute the missing model's weight proportionally to the remaining models. Previously, pandas sum(axis=1, skipna=True) silently dropped the NaN model's weight contribution, causing predictions to be systematically scaled down by ~35% for timestamps beyond the weather horizon. Now weights are reindexed to match predictions, zeroed where predictions are NaN, and the weighted sum is divided by the available weight total. When all models are NaN, the result is 0 (matching prior behavior). Includes regression test with seeded data verifying no NaN propagation and no systematic downscaling. Signed-off-by: Egor Dmitriev <[email protected]> Signed-off-by: Egor Dmitriev <[email protected]> * refactor(STEF-2854): extract nan_aware_weighted_mean helper Extract NaN-aware weight renormalization into a reusable helper in openstef_core.utils.pandas and use it in learned_weights_combiner. Removes type: ignore comments from _predict_quantile. Signed-off-by: Egor Dmitriev <[email protected]> * feat(STEF-2854): add skip_analysis param to BenchmarkPipeline.run() Allows skipping per-target and global analysis steps when running benchmarks. Useful when analysis will be run separately later via the comparison pipeline. Signed-off-by: Egor Dmitriev <[email protected]> * feat(STEF-2854): add filterings override to AnalysisConfig Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): resolve ruff lint warnings Signed-off-by: Egor Dmitriev <[email protected]> * fix(STEF-2854): resolve pyright type errors in modified files Signed-off-by: Egor Dmitriev <[email protected]> --------- Signed-off-by: Egor Dmitriev <[email protected]>
* add shifter transform and tests Signed-off-by: MentReeze <[email protected]> * added shifter to forecasting workflow Signed-off-by: MentReeze <[email protected]> * changed averaging to aggregated, simplified edge handling and removed cast to datetimeindex Signed-off-by: MentReeze <[email protected]> * changed to list of shifts Signed-off-by: MentReeze <[email protected]> * small fix: removed redundant list() Signed-off-by: MentReeze <[email protected]> --------- Signed-off-by: MentReeze <[email protected]>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.