Skip to content

Comments

[production/GFS.v17] Add zstandard restart option and update WW3 hash#3105

Open
dpsarmie wants to merge 5 commits intoufs-community:production/GFS.v17from
dpsarmie:v17/zstd_ww3_geom
Open

[production/GFS.v17] Add zstandard restart option and update WW3 hash#3105
dpsarmie wants to merge 5 commits intoufs-community:production/GFS.v17from
dpsarmie:v17/zstd_ww3_geom

Conversation

@dpsarmie
Copy link
Collaborator

Commit Queue Requirements:

  • This PR addresses a relevant WM issue (if not, create an issue).
  • All subcomponent pull requests (if any) have been reviewed by their code managers.
  • Run the full Intel+GNU RT suite (compared to current baselines), preferably on Ursa (Derecho or Hercules are acceptable alternatives). Exceptions: documentation-only PRs, CI-only PRs, etc.
    • Commit log file w/full results from RT suite run (if applicable).
    • Verify that test_changes.list indicates which tests, if any, are changed by this PR. Commit test_changes.list, even if it is empty.
  • Fill out all sections of this template.

Description:

This update brings in a couple of updates:

  1. Separate Zstandard options for the output and restarts: The project asked for the capability to set the zstandard compression options to the output and the restart separately. Currently, one option is applied to both. The changes in UFSATM allow for this option to now be controlled by the zstandard_level_rst and ideflate_rst options in the model_configure file. This was tested but will not be added to develop at this time.

  2. Update the MOM6 namelist with the WRTIE_GEOM variable:
    Changes made in add GUST_CONST as a variable  #3056 for WRITE_GEOM will be brought into production.

  3. Update the WW3 hash with the production branch:
    The hash for WW3 to point to the newly made production/GFS.v17 branch. This update also brings in EE2 changes that are already in develop (Sync WW3 develop to dev/ufs-weather-model #3091).

Commit Message:

* UFSWM - Update to get zstd options, EE2 changes, and variable definitions
  * UFSATM - Add option to control zstd compression in restart files separately from output
  * WW3 - Remove remarks and warnings / Create `production/GFS.v17` branch

Priority:

  • High: Needed for GFSv17 Production

Git Tracking

UFSWM:

No issues for production branch updates

Sub component Pull Requests:

UFSWM Blocking Dependencies:

  • None

Documentation:

  • Documentation update NOT required.

Changes

Regression Test Changes (Please commit test_changes.list):

  • No Baseline Changes.

Input data Changes:

  • None.

Library Changes/Upgrades:

  • No Updates

Testing Log:

  • RDHPCS
    • Orion
    • Hercules
    • GaeaC6
    • Derecho
    • Ursa
  • WCOSS2
    • Dogwood/Cactus
    • Acorn
  • CI
  • opnReqTest (complete task if unnecessary)

@JessicaMeixner-NOAA
Copy link
Collaborator

Thank you @dpsarmie !

@NickSzapiro-NOAA
Copy link
Collaborator

Have we activated these zstandard_level_rst and ideflate_rst options and tested debug, mpi, threading, restart reproducibility?

@dpsarmie
Copy link
Collaborator Author

Have we activated these zstandard_level_rst and ideflate_rst options and tested debug, mpi, threading, restart reproducibility?

I've tested these changes with the operational config for v17 and using RTs. There are some failures with RTs that have GOCART. I will be investigating this to see what the issue could be.

For all associated v17 tests, it runs fine and reproduces. These changes will not be for develop until the issue with non-v17 configurations are resolved (and if we decide that these options are needed).

The option will be turned off for v17 in the workflow for now, but the leads wanted this option available to do their own end-to-end tests in the workflow to make sure that the zstandard restarts work with DA, restarts, etc.

So long story short, this will still need to be tested in GW (which is why we're bringing in these changes now) but for the v17 related testing, it looks to be working and checking all the needed boxes. If it fails in GW, then the option will be turned off or even have the UFSATM/io changes reverted.

Copy link
Collaborator

@NickSzapiro-NOAA NickSzapiro-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, thanks @dpsarmie . I wasn't aware GW did reproducibility tests ... are these existing tests or TBD for this feature?

@dpsarmie
Copy link
Collaborator Author

There's GW testing of this branch that is ongoing. So I will merge this into production once I get the go ahead from the project leads.
Thanks everyone for your input! It's always appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Evaluating

Development

Successfully merging this pull request may close these issues.

5 participants