Add YAML linting and improve `setup` script by maddenp-cu · Pull Request #80 · NOAA-EPIC/EAGLE

maddenp-cu · 2026-03-05T07:11:40Z

Description:

Fixes #78 by adding yamllint to the base conda environment, adding a make yamllint target, incorporating that target into the test target, and documenting this in the README.md.

Also, improve the setup script by installing both run and dev packages in a single command to reduce the amount of time spent by conda in resolving package sets; and make it easy to install dev packages on top of a non-dev set of environments, or to change package versions, without removing conda or its environments.

Type of change:

New feature
Refactor / cleanup
Documentation

Area(s) affected

uwtools-based software environment and Makefile targets under src/

Commit Requirements:

This PR addresses a relevant NOAA-EPIC/EAGLE issue (if not, create an issue); a person responsible for submitting the update has been assigned to the issue (link issue)
Fill out all sections of this template.
I have performed a self-review of my own code
My changes generate no new warnings
I have made corresponding changes to the system documentation if necessary

Testing / Verification:

I ran and/or verified the changes (or provided a test plan)

I ran the Quickstart routine from the README.md.

Runtime Environment:

As specified in the README.md Quickstart, on Ursa.

maddenp-cu · 2026-03-05T07:14:03Z

src/config/base.yaml

@@ -1,5 +1,7 @@
 # This is the base EAGLE config. It currently configures the Nested EAGLE case.

+# yamllint disable rule:anchors rule:line-length


When final configs are created by composing multiple YAML files together, it may be that one config contains a YAML alias for which no corresponding anchor is apparently available -- but it will be available after composition.

EAGLE YAML configs may also have long lines.

maddenp-cu · 2026-03-05T07:14:36Z

src/Makefile

 	@(set -x && shellcheck --format=gcc --severity=info --shell=bash $(BASHSRCS))

-test: lint shellcheck typecheck
+test: lint shellcheck typecheck yamllint


make test will now also run make yamllint, though the latter can also be requested independently.

maddenp-cu · 2026-03-05T07:16:37Z

src/setup

-  # invoking make targets requires make itself; and driver execution and config management require
-  # uwtools, so install those.
-  conda create -y -q -c ufs-community -n $* jq $MAKE $UWTOOLS
+  conda create -y -q -c ufs-community -n $* $UWTOOLS


Only the base environment needs jq, and make needs to be available outside the conda installation, as it is used to create the conda installation. A note has been added to the README.md about the make requirement. This should be present, or easily made available, on just about any system.

maddenp-cu · 2026-03-05T07:18:06Z

src/setup

+    $UWTOOLS
+    yamllint=1.38.*
+  )
+  test -v EAGLE_DEV && pkgs+=( $(devpkgs) )


By conditionally appending the development packages to the list of packages to install, a single conda create or conda install command can be performed instead of two.

maddenp-cu · 2026-03-05T07:20:11Z

src/setup

+  if conda_env_exists $name; then
+    conda_install ${args[@]}
+  else
+    conda_create ${args[@]}


Given this logic, if e.g. make env ... is run once to create a non-dev environment, make devenv ... can later be run and conda (and pip) will find almost all required packages already installed, and will only need to install the missing dev packages. Or, a user could change version numbers here for experimental reasons, re-run make env, and update the existing conda environments without deleting them or the conda installation itself. This could be a big time saver.

AnilKumar-NOAA · 2026-03-05T16:21:04Z

src/Makefile

+yamllint:
+	$(call activate,base)
+	@echo "=> Linting YAML configs"
+	@(set -x && yamllint --no-warnings config/)
+


each recipe line runs in a separate shell. conda activate base line won’t affect the later yamllint. meaning yamllint will run in whatever environment Make happens to have.
possible suggestion
yamllint:
@echo "=> Linting YAML configs"
$(call activate,base) && (set -x && yamllint --no-warnings config/)

The .ONESHELL: directive higher in the Makefile ensures that all recipe lines run in the same shell.

yes. Thats great!

AnilKumar-NOAA · 2026-03-05T16:30:04Z

src/setup

+    python=3.13
+    xesmf=0.8.*


Note: may be fine on Ursa if you’ve validated solver availability, but python 3.13 + compiled geo stack (ESMF / esmpy / xesmf) is a common point of failure on HPC channels.

I haven't seen issues yet, but thanks for the heads-up.

AnilKumar-NOAA · 2026-03-05T16:49:12Z

src/setup

-  # invoking make targets requires make itself; and driver execution and config management require
-  # uwtools, so install those.
-  conda create -y -q -c ufs-community -n $* jq $MAKE $UWTOOLS
+  conda create -y -q -c ufs-community -n $* $UWTOOLS


conda_create to use -n "$name" with shift ?

Good idea, I think that would be clearer. I'll update and test. Thanks!

AnilKumar-NOAA · 2026-03-05T16:50:33Z

src/setup

-# shellcheck disable=1090,1091,2046,2048,2068,2086,2206,2155
+# shellcheck disable=1090,1091,2046,2048,2068,2086,2206,2207

 set -aeu


do we really need -a?
prefer set -euo pipefail

Yes, -a ensures that the functions defined at the top level are available in subshells.

I'll test with -o pipefail and either add it or address any issues. I remember having problems with it in ternary expressions like stmt && then || else, but maybe it doesn't matter here.

AnilKumar-NOAA

looks good to me!

AnilKumar-NOAA · 2026-03-05T17:15:15Z

src/setup

+  )
 }

 parse_kvargs $@


parse_kvargs should use "$@"?? (both in the call and in the loop).

In the general case I agree, because the individual arguments in $@ could in theory contain spaces. But the only arguments allowed here are of the form key=value, with no spaces. In my opinion, having "$@" suggests to readers that we think that there might be spaces, which isn't true. I think using syntax choices to communicate intent is useful and so am reluctant to add more syntax where it isn't needed.

But I might be missing something -- please let me know if so!

Its fine - will see if any issues later.

maddenp-cu added 13 commits March 4, 2026 21:56

Updates from vx-env-split branch

2874a62

Add yamllint etc.

24edc3f

WIP

e151dc3

WIP

3a89302

WIP

c4dc4d5

WIP

42854e5

WIP

12191f4

WIP

326c8a3

WIP

3611273

Filter out 'Requirement already satisfied:' pip messages

8c78e4e

shellcheck advice

22ad7ca

WIP

a507edf

Merge branch 'main' into yamllint

8e1b9c1

maddenp-cu commented Mar 5, 2026

View reviewed changes

maddenp-cu marked this pull request as ready for review March 5, 2026 16:09

maddenp-cu requested a review from a team as a code owner March 5, 2026 16:09

maddenp-cu requested a review from elcarpenterNOAA March 5, 2026 16:09

AnilKumar-NOAA reviewed Mar 5, 2026

View reviewed changes

Address PR feedback

d5763a3

elcarpenterNOAA approved these changes Mar 5, 2026

View reviewed changes

AnilKumar-NOAA approved these changes Mar 5, 2026

View reviewed changes

maddenp-cu merged commit 879adc5 into NOAA-EPIC:main Mar 5, 2026
27 of 29 checks passed

maddenp-cu deleted the yamllint branch March 5, 2026 19:49

		@@ -1,5 +1,7 @@
		# This is the base EAGLE config. It currently configures the Nested EAGLE case.

		# yamllint disable rule:anchors rule:line-length

Conversation

maddenp-cu commented Mar 5, 2026

Description:

Type of change:

Area(s) affected

Commit Requirements:

Testing / Verification:

Runtime Environment:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnilKumar-NOAA Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnilKumar-NOAA Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnilKumar-NOAA left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maddenp-cu Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AnilKumar-NOAA Mar 5, 2026 •

edited

Loading

AnilKumar-NOAA Mar 5, 2026 •

edited

Loading

AnilKumar-NOAA left a comment •

edited

Loading

maddenp-cu Mar 5, 2026 •

edited

Loading