NGWPC NWM Regionalization, Hindcast, and Coastal Tools 2026-06-09

[cmaynard-ngwpc]

Delivered a major modernization of the RTE workflow, development environment, testing framework, forecasting capabilities, and deployment tooling. Expanded forecast support with lagged ensembles, MPI-enabled execution, default realizations, cold starts, calibration-driven forecasts, LSTM integration, NWM output variables, configurable forcing products, and improved handling of forecast, calibration, and testing workflows. Refactored forecast execution into dedicated scripts and configuration models, added support for IceFabric and NHF hydrofabrics, improved forecast configuration management, and enhanced forecast validation and testing. Significantly expanded automated testing with new unit, integration, calibration, forecast, and execution test frameworks, GitHub Actions support, pytest integration, test result analysis tools, and automated workflow validation. Modernized developer tooling through extensive Dev Container, debugger, documentation, MkDocs, CI/CD, and build-system enhancements, while improving package sourcing flexibility, Docker image management, GitHub workflow automation, Slurm integration, multi-user support, and reproducible deployments. Additional improvements included enhanced logging, data setup automation, WCOSS compatibility, regionalization support, calibration workflow refactoring, code cleanup, CLI usability improvements, dependency management updates, and extensive documentation and user-guidance enhancements throughout the project.

Additions

• Expanded automated testing coverage, expected-result data, unit-test support, tolerance-based comparisons, serialization checks, and CI-exercised validation paths.
• Added and expanded forcing workflow support, including updated configuration templates, forecast product handling, lagged-ensemble inputs, regional forcing behavior, and package-delivered forcing resources.
• Added NHF, hydrofabric, geopackage, VPU, and crosswalk workflow capabilities, including utility scripts, oCONUS handling, and domain/gage data integration.
• Implemented and expanded serialization support, including serialized size metadata, message constants, integer-based transfer structures, deserialization bounds handling, and state-transfer messaging.
• Added state saving, state restoration, reset-time, restart, and hindcast workflow capabilities, including related time-management updates and model restart handling.
• Expanded verification and evaluation capabilities with additional metric calculation behavior, observation retrieval support, plotting updates, gage-source integration, and forecast-to-observation alignment.
• Integrated EWTS logging support across affected components, including module-specific logging updates, optional logging behavior, Docker/build integration, and unit-test coverage where applicable.
• Added additional release updates covering updates to run_region.sh for new way of calling run_regionalization.py as a module instead of as a script, updates to example shell scripts for new method of running as module, additions for hydrofab_file CLI arg to enable icefabric bypass, Enable EDFS streamflow download, and additions for optional timestamp to fcst_run_name.

Removals

• Removed unused, duplicate, obsolete, and deprecated code paths, including stale helper functions, duplicated definitions, older EWTS references, and no-longer-needed implementation artifacts.
• Removed additional release updates covering removal of CLI arg --hydrofab_file from run_forecast.py, removal of TODO, removal of env var export, removal of pseudocode, and removal of rm -rf calls from high level example scripts.

Changes

• Updated cache, dataset, temporary-file, scratch-directory, and data-access behavior, including lazy loading, explicit dataset closure, retry handling, and local/S3 access refinements.
• Refined configuration, realization, template, sample-config, and package-resource handling across supported workflows and deployment environments.
• Updated CI/CD, security scanning, downstream pipeline triggers, and customer-delivery workflow configuration.
• Updated Docker, build, packaging, dependency, and installed-resource configuration.
• Updated geospatial metadata, regridding, CRS, mesh/geogrid, domain, gage, and crosswalk handling across forcing, verification, and realization workflows.
• Improved validation, warning, exception, traceback, permission, and user-facing error-reporting behavior.
• Refactored and cleaned up implementation details, including naming consistency, type hints, imports, property handling, and internal helper structure.
• Updated hydrologic model outputs and model-specific runtime behavior, including formulation outputs, GIUH/channel-inflow values, ponded-depth variables, and model-specific state/runtime handling.
• Reworked serialization behavior, message metadata, reported state types, size calculations, buffer validation, and transfer logic for state save and restore workflows.
• Refactored BMI model interfaces and input-forcing behavior, including model-class selection, getter/setter behavior, bounds handling, and BMI calculation fixes.
• Updated additional release updates covering updates to docs, Docstrings, Relocate Alaska NWM spatial meta file and improve data setup, move chmod to be after ensure_dir in run_region.sh, and run_region.sh: extend container's existing PYTHONPATH.