Write GSI diagnostic files directly to DATAROOT to eliminate tar/untar overhead

[Copilot]

Rework GSI Diagnostic File Writes Using dataroot_com_path

Changes Implemented

Based on feedback:

• Use dataroot_com_path function from feature/dataroot_util branch
• Generate pCOM_ATMOS_ANALYSIS from COM_ATMOS_ANALYSIS (not hardcoded path)
• Link pCOM_ATMOS_ANALYSIS/gsidiags into working directory
• No moving of directories - GSI writes directly through symlink
• Remove all tar/untar operations

Modified Files:

1. ush/bash_utils.sh - Added dataroot_com_path function from feature/dataroot_util
2. dev/jobs/JGLOBAL_ATMOS_ANALYSIS - Generate pCOM_ATMOS_ANALYSIS using dataroot_com_path
3. dev/jobs/JGLOBAL_ATMOS_ANALYSIS_DIAG - Generate pCOM_ATMOS_ANALYSIS for reading
4. dev/scripts/exglobal_atmos_analysis.sh:
   • Create gsidiags directory in pCOM before GSI runs
   • Symlink ${DATA}/gsidiags -> ${pCOM_ATMOS_ANALYSIS}/gsidiags
   • Change to gsidiags directory before running GSI
   • Remove tar creation and copy to COM
5. dev/scripts/exglobal_diag.sh:
   • Symlink gsidiags from pCOM to DATA
   • Change to gsidiags directory for processing
   • Remove tar copy and extraction

Key Benefits:

• GSI writes dir.???? directly to persistent DATAROOT location
• No tar/untar overhead
• Downstream jobs access files via same symlink pattern
• Uses approved dataroot_com_path utility

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Rework gsi diagnostic file writes to write to DATAROOT</issue_title>
<issue_description>### What new functionality do you need?

NCO has approved a method of writing files directly to DATAROOT. The first target for this procedure will be the gsidiags files being written by gsi.x and pushed to COM.

Acceptance Criteria

• gsidiags files are written directly to DATAROOT/...
• The diagnostic files are no longer tarred and sent to COM
• All downstream jobs run as expected with zero-diff changes
• The gdas_anal, gfs_anal, and enkfgdas_eobs jobs run faster

Suggest a solution (optional)

1. Use the utility proposed in Add a function to create COM-like paths under DATAROOT NOAA-EMC/global-workflow#4476 to create the pCOM_ATMOS_ANLYSIS variable to point to DATAROOT in JGLOBAL_ATMOS_ANALYSIS
2. Create ${pCOM_ATMOS_ANALYSIS}/gsidiags and link it to ${DATA} in exglobal_atmos_analysis.sh
3. Remove the tarring and copying currently in exglobal_atmos_analysis.sh, lines 898-907
4. Remove copying and untarring currently in exglobal_diag.sh, lines 42, 59-65
5. Use the utility proposed in Add a function to create COM-like paths under DATAROOT NOAA-EMC/global-workflow#4476 to create the pCOM_ATMOS_ANALYSIS variable to point to the same location in DATAROOT in JGLOBAL_ATMOS_DIAG
6. Create a link from ${pCOM_ATMOS_ANALYSIS}/gsidiags to ${DATA} in exglobal_diag.sh
7. Allow the diagnostic scripts/executables to read directly from the linked directory</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Rework gsi diagnostic file writes to write to DATAROOT NOAA-EMC/global-workflow#4479

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[DavidHuber-NOAA]

@copilot please rework this by

• using the feature/dataroot_util branch (https://github.com/DavidHuber-NOAA/global-workflow/tree/feature/dataroot_util) to generate pCOM_ATMOS_ANALYSIS from COM_ATMOS_ANALYSIS
• not moving directories at all. Instead, link pCOM_ATMOS_ANALYSIS/gsidiags into the working directory

[Copilot]

@copilot please rework this by

• using the feature/dataroot_util branch (https://github.com/DavidHuber-NOAA/global-workflow/tree/feature/dataroot_util) to generate pCOM_ATMOS_ANALYSIS from `COM_A...

I've reworked the implementation to address both points:

1. Using dataroot_com_path from feature/dataroot_util: The dataroot_com_path function is now integrated and used to generate pCOM_ATMOS_ANALYSIS from the existing COM paths (rather than hardcoding the path structure).

2. No moving directories - using symlinks: GSI now writes directly to the persistent location via symlinks:

   • Before GSI runs: Create ${pCOM_ATMOS_ANALYSIS}/gsidiags and symlink it as ${DATA}/gsidiags
   • Change to the gsidiags directory so GSI writes dir.???? there
   • The diag job accesses the same location via symlink

Changes in commit 3b3d04d:

• Added dataroot_com_path function to ush/bash_utils.sh
• Job files use pCOM_ATMOS_ANALYSIS=$(dataroot_com_path "${COMOUT_ATMOS_ANALYSIS}")
• GSI diagnostic directories written directly to persistent DATAROOT location
• All tar/untar operations removed
• All shellcheck validations pass

Note: The local commit diverges from the remote history (which contains the previous implementation). A force push would be needed to update the PR.