Refactor Local Archiving: Move YAML Logic to archive_vars.py and Handle Ensemble/Deterministic Archiving

[AntonMFernando-NOAA]

Description

• The archiving process includes two separate components: local archiving (arch_vrfy) and external tar creation (arch_tar). This issue focuses only on local archiving.

This PR will handle:

• Archiving of ensemble output files

• Archiving of deterministic (control) output files

• Logic, COM paths, and cycle-dependent variables currently defined in the YAML files ( ***_arcdir.yaml.j2) will be moved into archive_vars.py. The YAML files will retain only the file-specific archiving definitions.

  Resolves: Archive Enhancement — Migrate Template Variables from Jinja to Python #3910
  Refs: [Enabler] Review the archiving process and determine best paths forward #3656

Type of change

• Bug fix (fixes something broken)
• New feature (adds functionality)
• Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

• Is this a breaking change (a change in existing functionality)? NO
• Does this change require a documentation update? NO
• Does this change require an update to any of the following submodules? NO
  • EMC verif-global
  • GDAS
  • GFS-utils
  • GSI
  • GSI-monitor
  • GSI-utils
  • UFS-utils
  • UFS-weather-model
  • wxflow

How has this been tested?

• CI tests on hera

Checklist

• Any dependent changes have been merged and published
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have documented my code, including function, input, and output descriptions
• My changes generate no new warnings
• New and existing tests pass with my changes
• This change is covered by an existing CI test or a new one has been added
• Any new scripts have been added to the .github/CODEOWNERS file with owners
• I have made corresponding changes to the system documentation if necessary

[DavidHuber-NOAA]

Looks good overall. Definitely like the cleanup happening in the YAMLs. I have just one issue on how templates are being rendered.

Also, I'm wondering if the arcdir templates should be split up by RUN rather than NET to remove the if RUN == ... business. I don't think that needs to be part of this issue, though.

[aerorahul]

Looks good. Just a couple of additional suggestions.

[TravisElless-NOAA]

If this is a copy_opt should this comment say this files are optional or is the comment correct that these are required files and should be copy_req?

[AntonMFernando-NOAA]

Yes. Removed Required.

[AntonMFernando-NOAA]

@DavidHuber-NOAA @aerorahul All the tests checked. You can trigger off the CI tests now.

[aerorahul]

lgtm

[DavidHuber-NOAA]

All tests passed on WCOSS2. Merging.