Skip to content

Adding GCAFS products v2 #4407

Merged
DavidHuber-NOAA merged 8 commits intoNOAA-EMC:developfrom
bbakernoaa:feature/gcafsproductsv2
Jan 13, 2026
Merged

Adding GCAFS products v2 #4407
DavidHuber-NOAA merged 8 commits intoNOAA-EMC:developfrom
bbakernoaa:feature/gcafsproductsv2

Conversation

@bbakernoaa
Copy link
Contributor

Description

This pull request updates the configuration and product parameter lists for the GCAFS workflow, focusing on standardizing and expanding the set of output variables, improving product consistency, and updating archiving logic. The changes include major updates to parameter lists, modifications to downset settings, and enhancements to archive templates.

Key changes include:

Product parameter list updates:

  • Added new parameter lists gcafs.anl.paramlist.b.txt and gcafs.f000.paramlist.b.txt, specifying a comprehensive set of fields and levels for GCAFS analysis and forecast products.
  • Overhauled gcafs.fFFF.paramlist.b.txt: removed many obsolete or less-used fields and levels, and replaced them with a new, more focused set, particularly for PMTC and PMTF variables across a wide range of pressure levels.

Configuration changes:

  • Updated config.atmos_products to set downset=2 for both gcdas and gcafs runs (was previously 1), and added new environment variables for the new parameter lists (paramlistb_anl, paramlistb_f000). [1] [2]

Archiving and dataset template updates:

  • Enhanced master_gcafs.yaml.j2 to re-enable and expand archiving of pres_b GRIB2 products at 0.25, 0.5, and 1.0 degree resolutions.
  • Updated master_gcdas.yaml.j2 to add and comment out several new analysis and sub-sampled GRIB2 datasets for future archiving, and to include new netCDF analysis files.

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? YES/NO (If YES, please indicate to which system(s))
    • GFS
    • GEFS
    • SFS
    • GCAFS
  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

GAEA

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

bbakernoaa and others added 3 commits January 2, 2026 16:11
….paramlist.b.txt

- Removed redundant and obsolete parameters from both files to streamline data processing.
- Added new parameters to enhance atmospheric modeling capabilities, including various pressure levels and hybrid levels.
- Consolidated entries for clarity and improved organization, ensuring consistency across parameter definitions.
…amlist.a.txt, gcafs.fFFF.paramlist.a.txt, and gcafs.fFFF.paramlist.b.txt to remove redundant entries and maintain consistency
@bbakernoaa bbakernoaa marked this pull request as ready for review January 6, 2026 16:28
@bbakernoaa bbakernoaa self-assigned this Jan 6, 2026
aerorahul
aerorahul previously approved these changes Jan 6, 2026
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks straightforward. approved pending CI testing.

@bbakernoaa
Copy link
Contributor Author

@aerorahul thanks!

DavidHuber-NOAA
DavidHuber-NOAA previously approved these changes Jan 9, 2026
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks @bbakernoaa!

@DavidHuber-NOAA
Copy link
Contributor

@bbakernoaa I see now that there is a conflict in parm/archive/master_gcdas.yaml.j2. Could you merge develop and resolve that conflict? After that, I will launch tests on C6.

@bbakernoaa bbakernoaa dismissed stale reviews from DavidHuber-NOAA and aerorahul via 4b97018 January 9, 2026 18:40
@DavidHuber-NOAA
Copy link
Contributor

Launching on C6.

@emcbot emcbot added CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 and removed CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 labels Jan 9, 2026
@emcbot
Copy link

emcbot commented Jan 9, 2026

BUILD FAILED on Gaeac6 (pipeline ID: 6982)

In directory: /gpfs/f6/drsa-precip3/world-shared/global/CI/GITLAB/pr_cases_4407_4e8e5a4f_6982/global-workflow/sorc

Error log files:


/gpfs/f6/drsa-precip3/world-shared/global/CI/GITLAB/pr_cases_4407_4e8e5a4f_6982/global-workflow/sorc/logs/gsi_enkf.log

View Error Logs

This failure was detected automatically by global-workflow's CI/CD Pipeline

@emcbot emcbot added CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed and removed CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 labels Jan 9, 2026
@DavidHuber-NOAA
Copy link
Contributor

Builds on C6 have been running slow this week, perhaps due to C5 maintenance. I will launch the PR on WCOSS2 instead.

@DavidHuber-NOAA DavidHuber-NOAA added CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress and removed CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed labels Jan 9, 2026
@DavidHuber-NOAA
Copy link
Contributor

I will verify that the archiving jobs ran correctly on Dogwood after the h2 filesystem comes back online.

@DavidHuber-NOAA
Copy link
Contributor

I enabled GCAFS archiving on Dogwood and verified that all archiving jobs ran to completion without error. Merging.

@CoryMartin-NOAA
Copy link
Contributor

Thank you @DavidHuber-NOAA !

@DavidHuber-NOAA DavidHuber-NOAA merged commit 5091a7b into NOAA-EMC:develop Jan 13, 2026
5 checks passed
@bbakernoaa bbakernoaa deleted the feature/gcafsproductsv2 branch January 13, 2026 16:11
@ypwang19 ypwang19 mentioned this pull request Jan 14, 2026
21 tasks
CoryMartin-NOAA pushed a commit that referenced this pull request Jan 14, 2026
# Description
This PR turns on the archiving of aerosol analysis b products, given
that GCAFS/GCDAS analysis b products are enabled in a recent PR
#4407.

# Type of change
- [ ] New feature (adds functionality)

# Change characteristics
- Is this change expected to change outputs (e.g. value changes to
existing outputs, new files stored in COM, files removed from COM,
filename changes, additions/subtractions to archives)? YES
  - [ ] GCAFS
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO
- Does this change require an update to any of the following submodules?
NO
  - [ ] EMC verif-global <!-- NOAA-EMC/EMC_verif-global#1234 -->
  - [ ] GDAS <!-- NOAA-EMC/GDASApp#1234 -->
  - [ ] GFS-utils <!-- NOAA-EMC/gfs-utils#1234 -->
  - [ ] GSI <!-- NOAA-EMC/GSI#1234 -->
  - [ ] GSI-monitor <!-- NOAA-EMC/GSI-Monitor#1234 -->
  - [ ] GSI-utils <!-- NOAA-EMC/GSI-Utils#1234 -->
  - [ ] UFS-utils <!-- ufs-community/UFS_UTILS#1234 -->
  - [ ] UFS-weather-model <!-- ufs-community/ufs-weather-model#1234 -->
  - [ ] wxflow <!-- NOAA-EMC/wxflow#1234 -->

# How has this been tested?
- Cycled C96_gcafs_cycled on Hera

# Checklist
- [ ] Any dependent changes have been merged and published
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have documented my code, including function, input, and output
descriptions
- [ ] My changes generate no new warnings
- [ ] New and existing tests pass with my changes
- [ ] This change is covered by an existing CI test or a new one has
been added
- [ ] Any new scripts have been added to the .github/CODEOWNERS file
with owners
- [ ] I have made corresponding changes to the system documentation if
necessary
weihuang-jedi added a commit to NOAA-EPIC/global-workflow-cloud that referenced this pull request Jan 22, 2026
…lobal-workflow-cloud into feature/use_container_prefix

* 'feature/use_container_prefix' of github.com:NOAA-EPIC/global-workflow-cloud:
  sync with EMC build_all.sh fix
  Fix 06z gfs_sfcanl jobs failing due to a bug in CCPP (NOAA-EMC#4389)
  Build on compute or login nodes from one build script. (NOAA-EMC#4380)
  Refactor build_compute.py to optionally support node exclusion (NOAA-EMC#4447)
  Fix incorrect branching in bash code anl (NOAA-EMC#4442)
  Update of GitHub Copilot Instructions file for current MCP/RAG system (NOAA-EMC#4397)
  Fix reviewdog arg name (NOAA-EMC#4441)
  Fix bash code analysis issues (NOAA-EMC#4440)
  Enable gcafs tests on Ursa (NOAA-EMC#4439)
  Initialize LETKF solver after running observer (NOAA-EMC#4424)
  Move C96mx025 case to sfsv1 (NOAA-EMC#4435)
  Update HPC support levels. (NOAA-EMC#4430)
  Adding GCAFS products v2  (NOAA-EMC#4407)
  Feature/task template (NOAA-EMC#4427)
  Run GEMPAK jobs on prepost nodes (NOAA-EMC#4411)
  Build the GDASApp with 8 cores on Gaea again
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants