Skip to content

Test rocotoboot dryrun #4245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TerrenceMcGuinness-NOAA wants to merge 13 commits into from
+140 −4
Closed

Test rocotoboot dryrun #4245

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
6f912c8
Forcing the GitLab pipeline to use the Rocoto dryrun build over the s…
TerrenceMcGuinness-NOAA Nov 13, 2025
efedf2d
added UTIL path on ci config for platform specifics to custom rocoto …
TerrenceMcGuinness-NOAA Nov 13, 2025
53d995e
updated path to Rocoto used in pipeline
TerrenceMcGuinness-NOAA Nov 17, 2025
49541a2
Merge branch 'NOAA-EMC:develop' into test_rocotoboot_dryrun
TerrenceMcGuinness-NOAA Nov 17, 2025
de087c3
added paths to rocotoboot dryrun uddate for Ursa in ci platforms conf…
TerrenceMcGuinness-NOAA Nov 17, 2025
cc5470f
Merge branch 'NOAA-EMC:develop' into test_rocotoboot_dryrun
TerrenceMcGuinness-NOAA Nov 18, 2025
85b1d44
Merge branch 'NOAA-EMC:develop' into test_rocotoboot_dryrun
TerrenceMcGuinness-NOAA Dec 1, 2025
683baf0
Merge branch 'NOAA-EMC:develop' into test_rocotoboot_dryrun
TerrenceMcGuinness-NOAA Dec 3, 2025
9928ed1
needed to update PATH to pickup the rocotoboot we are testing
TerrenceMcGuinness-NOAA Dec 3, 2025
198b249
forgot to get MACHINE_ID to soruce the correct platform for getting t…
TerrenceMcGuinness-NOAA Dec 3, 2025
f93a587
needed to drop the GW on HOMEgfs in execure ctest template
TerrenceMcGuinness-NOAA Dec 3, 2025
df973aa
Merge branch 'NOAA-EMC:develop' into test_rocotoboot_dryrun
TerrenceMcGuinness-NOAA Dec 5, 2025
5196edd
Merge branch 'NOAA-EMC:develop' into test_rocotoboot_dryrun
TerrenceMcGuinness-NOAA Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .gitlab-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,18 @@ include:

echo "Setting up build environment for ${machine^}..."
echo "Using build directory ${GW_HOMEgfs}"

# TEMPORARY: Use custom Rocoto build with dryrun feature for testing
# TODO: Remove this when upstream Rocoto PR is merged
# Load platform-specific configuration first
source "${HOMEgfs}/dev/ci/platforms/config.${MACHINE_ID}"
if [[ -n "${ROCOTO_TEST_PATH:-}" && -d "${ROCOTO_TEST_PATH}" ]]; then
export PATH="${ROCOTO_TEST_PATH}:$PATH"
echo "Using custom Rocoto build at: $(which rocotorun 2>/dev/null || echo 'NOT FOUND')"
else
echo "ROCOTO_TEST_PATH not set or directory not found, using system Rocoto"
fi

git submodule status
# When GFS_CI_RUN_TYPE is set to "nightly", we use the develop branch
# It should be set when configuring the scheculed nightly pipeline
Expand Down
12 changes: 12 additions & 0 deletions dev/ci/gitlab-ci-cases.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,18 @@
script:
- |
echo "Using build directory ${GW_HOMEgfs} (dated ${DATE})"

# TEMPORARY: Use custom Rocoto build with dryrun feature for testing
# TODO: Remove this when upstream Rocoto PR is merged
# Load platform-specific configuration first
source "${GW_HOMEgfs}/dev/ci/platforms/config.${machine}"
if [[ -n "${ROCOTO_TEST_PATH:-}" && -d "${ROCOTO_TEST_PATH}" ]]; then
export PATH="${ROCOTO_TEST_PATH}:$PATH"
echo "Using custom Rocoto build at: $(which rocotorun 2>/dev/null || echo 'NOT FOUND')"
else
echo "ROCOTO_TEST_PATH not set or directory not found, using system Rocoto"
fi

pslot=$("${GW_HOMEgfs}/dev/ci/scripts/utils/ci_utils.sh" get_pslot "${RUNTESTS_DIR}" "${caseName}")
"${GW_HOMEgfs}/dev/ci/scripts/run_check_gitlab_ci.sh" "${GW_RUN_PATH}" "${pslot}" global-workflow
exit $?
Expand Down
22 changes: 22 additions & 0 deletions dev/ci/gitlab-ci-ctests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,18 @@
stage: setup_tests
script: |
echo "Setting up for ctests pipeline building in ${GW_HOMEgfs} on ${machine}"

# TEMPORARY: Use custom Rocoto build with dryrun feature for testing
# TODO: Remove this when upstream Rocoto PR is merged
# Load platform-specific configuration first
source "${GW_HOMEgfs}/dev/ci/platforms/config.${machine}"
if [[ -n "${ROCOTO_TEST_PATH:-}" && -d "${ROCOTO_TEST_PATH}" ]]; then
export PATH="${ROCOTO_TEST_PATH}:$PATH"
echo "Using custom Rocoto build at: $(which rocotorun 2>/dev/null || echo 'NOT FOUND')"
else
echo "ROCOTO_TEST_PATH not set or directory not found, using system Rocoto"
fi

source "${GW_HOMEgfs}/dev/ush/gw_setup.sh"
source "${GW_HOMEgfs}/dev/ci/platforms/config.${machine}"
cd "${GW_HOMEgfs}/dev/ctests" || exit 1
Expand All @@ -52,7 +64,17 @@
stage: run_tests
script: |
echo "Using build directory ${GW_HOMEgfs} for running CTests"

source "${GW_HOMEgfs}/dev/ush/gw_setup.sh"
# Platform config loaded by gw_setup.sh, now check for custom Rocoto
# TEMPORARY: Use custom Rocoto build with dryrun feature for testing
# TODO: Remove this when upstream Rocoto PR is merged
if [[ -n "${ROCOTO_TEST_PATH:-}" && -d "${ROCOTO_TEST_PATH}" ]]; then
export PATH="${ROCOTO_TEST_PATH}:$PATH"
echo "Using custom Rocoto build at: $(which rocotorun 2>/dev/null || echo 'NOT FOUND')"
else
echo "ROCOTO_TEST_PATH not set or directory not found, using system Rocoto"
fi
cd "${GW_HOMEgfs}/dev/ctests/build" || exit 1
echo "Running CTests with label: ${CTEST_NAME} in ${PWD}"
num=$(ctest -N | grep "Total Tests" | awk "{print \$3}")
Expand Down
13 changes: 13 additions & 0 deletions dev/ci/platforms/config.gaeac6
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,16 @@ export max_concurrent_pr=4 # number of concurrent pull requests

# HPC account information - For job submission
export HPC_ACCOUNT=${HPC_ACCOUNT:-drsa-precip3}

#########################################################################
# System-specific settings
# Platform-specific paths and utilities for CI/CD operations
#########################################################################

# Utilities path for custom builds and role account tools
# TODO: Populate with actual role account path for this machine
export UTIL_PATH="/path/to/role/account/utils"

# Custom Rocoto build path for testing (temporary)
# TODO: Remove when upstream Rocoto PR is merged
export ROCOTO_TEST_PATH="${UTIL_PATH}/Rocoto/bin"
13 changes: 13 additions & 0 deletions dev/ci/platforms/config.hera
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,16 @@ export max_concurrent_pr=4

# HPC account which overides the default account
export HPC_ACCOUNT=${HPC_ACCOUNT:-fv3-cpu}

#########################################################################
# System-specific settings
# Platform-specific paths and utilities for CI/CD operations
#########################################################################

# Utilities path for custom builds and role account tools
# TODO: Populate with actual role account path for this machine
export UTIL_PATH="/path/to/role/account/utils"

# Custom Rocoto build path for testing (temporary)
# TODO: Remove when upstream Rocoto PR is merged
export ROCOTO_TEST_PATH="${UTIL_PATH}/Rocoto/bin"
15 changes: 14 additions & 1 deletion dev/ci/platforms/config.hercules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,17 @@ export GITLAB_BUILDS_DIR=${GFS_CI_ROOT}/BUILDS/GITLAB
export GITLAB_RUNNER_DIR="${GFS_CI_ROOT}/GitLab/Runner"

# HPC account which overides the default account
export HPC_ACCOUNT=${HPC_ACCOUNT:-fv3-cpu}
export HPC_ACCOUNT=${HPC_ACCOUNT:-epic-ps}
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably want to revert this to fv3-cpu before merging. Fine for testing.


#########################################################################
# System-specific settings
# Platform-specific paths and utilities for CI/CD operations
#########################################################################

# Utilities path for custom builds and role account tools
# TODO: Populate with actual role account path for this machine
export UTIL_PATH="/path/to/role/account/utils"

# Custom Rocoto build path for testing (temporary)
# TODO: Remove when upstream Rocoto PR is merged
export ROCOTO_TEST_PATH="${UTIL_PATH}/Rocoto/bin"
13 changes: 13 additions & 0 deletions dev/ci/platforms/config.orion
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,16 @@ export GITLAB_RUNNER_DIR="${GFS_CI_ROOT}/GitLab/Runner"

# HPC account which overides the default account
export HPC_ACCOUNT=${HPC_ACCOUNT:-fv3-cpu}

#########################################################################
# System-specific settings
# Platform-specific paths and utilities for CI/CD operations
#########################################################################

# Utilities path for custom builds and role account tools
# TODO: Populate with actual role account path for this machine
export UTIL_PATH="/path/to/role/account/utils"

# Custom Rocoto build path for testing (temporary)
# TODO: Remove when upstream Rocoto PR is merged
export ROCOTO_TEST_PATH="${UTIL_PATH}/Rocoto/bin"
13 changes: 13 additions & 0 deletions dev/ci/platforms/config.ursa
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,16 @@ export max_concurrent_pr=4

# HPC account which overides the default account
export HPC_ACCOUNT=${HPC_ACCOUNT:-fv3-cpu}

#########################################################################
# System-specific settings
# Platform-specific paths and utilities for CI/CD operations
#########################################################################

# Utilities path for custom builds and role account tools
# TODO: Populate with actual role account path for this machine
export UTIL_PATH="/path/to/role/account/utils"

# Custom Rocoto build path for testing (temporary)
# TODO: Remove when upstream Rocoto PR is merged
export ROCOTO_TEST_PATH="${UTIL_PATH}/Rocoto/bin"
14 changes: 14 additions & 0 deletions dev/ci/platforms/config.wcoss2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,20 @@

export GFS_CI_ROOT=/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT
export ICSDIR_ROOT=/lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR
# HPC account which overides the default account
export HPC_ACCOUNT=${HPC_ACCOUNT:-GFS-DEV}

#########################################################################
# System-specific settings
# Platform-specific paths and utilities for CI/CD operations
#########################################################################

# Utilities path for custom builds and role account tools
# TODO: Populate with actual role account path for this machine
export UTIL_PATH="/path/to/role/account/utils"

# Custom Rocoto build path for testing (temporary)
# TODO: Remove when upstream Rocoto PR is merged
export ROCOTO_TEST_PATH="${UTIL_PATH}/Rocoto/bin"
export max_concurrent_cases=5
export max_concurrent_pr=4
4 changes: 2 additions & 2 deletions dev/ctests/scripts/execute.sh.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ TEST_NAME=$("${HOMEgfs}/dev/ci/scripts/utils/ci_utils.sh" get_pslot "@CMAKE_CURR
JOB=${2:?"Job name is required"}
idate=$3

rocotoboot_dryrun=$(command -v rocotoboot_dryrun) || true
# Use standard rocotoboot with --dryrun flag
CASEDIR="@CMAKE_CURRENT_BINARY_DIR@/RUNTESTS/EXPDIR/${TEST_NAME}"
cd "${CASEDIR}"
rm -f ./*.db
rm -f ./jobcard

yes | "${rocotoboot_dryrun}" --dryrun -d "${TEST_NAME}.db" -w "${TEST_NAME}.xml" -v 10 -c "${idate}00" -t "${JOB}" 2> jobcard || true
yes | rocotoboot --dryrun -d "${TEST_NAME}.db" -w "${TEST_NAME}.xml" -v 10 -c "${idate}00" -t "${JOB}" 2> jobcard || true
# Rocoto outputs the job card inside of {{}}. Strip all other output to get just the job card.
sed -n '/^{{/,/^}}/p' < jobcard | sed '1d;$d' > "${TEST_NAME}.sub" || true

Expand Down