Skip to content

Test rocotoboot dryrun#4245

Closed
TerrenceMcGuinness-NOAA wants to merge 13 commits intoNOAA-EMC:developfrom
TerrenceMcGuinness-NOAA:test_rocotoboot_dryrun
Closed

Test rocotoboot dryrun#4245
TerrenceMcGuinness-NOAA wants to merge 13 commits intoNOAA-EMC:developfrom
TerrenceMcGuinness-NOAA:test_rocotoboot_dryrun

Conversation

@TerrenceMcGuinness-NOAA
Copy link
Collaborator

Force CI pipelines to use local Rocoto build with dryrun feature

Summary

Implements temporary changes to force all CI/CD pipelines to use a local build of Rocoto with the dryrun feature for comprehensive stress testing across all HPC platforms. Uses configurable platform-specific paths for role account flexibility.

Resolves #4220

Changes Made

  • CI Pipeline Templates: Added configurable PATH override using platform-specific settings
  • Platform Configurations: Added system-specific settings section to all platform config files
  • CTest Framework: Updated execute.sh.in to use standard rocotoboot --dryrun instead of custom binary
  • Comprehensive Coverage: Affects build processes, experiment runs, and CTest execution

Files Modified

  • .gitlab-ci.yml - Build template with configurable PATH override
  • dev/ci/gitlab-ci-cases.yml - Experiment run template with configurable PATH override
  • dev/ci/gitlab-ci-ctests.yml - CTest template with configurable PATH override
  • dev/ctests/scripts/execute.sh.in - Standard rocotoboot command usage
  • dev/ci/platforms/config.* - Added system-specific settings sections (7 files)

Technical Approach

Uses configurable platform-specific paths from role account settings:

# In platform config files
export UTILS_PATH="/path/to/role/account/utils"
export ROCOTO_TEST_PATH="${TEST_UTILS_PATH}/Rocoto/bin"

# In CI templates
if [[ -n "${ROCOTO_TEST_PATH:-}" && -d "${ROCOTO_TEST_PATH}" ]]; then
  export PATH="${ROCOTO_TEST_PATH}:$PATH"
fi

NOTE: Draft PR until all local Rocoto installs are in place and paths above are updated appropriately

Platform Configuration

Added "System-specific settings" section to all platform configs with:

  • UTILS_PATH - Base path for role account utilities
  • ROCOTO_TEST_PATH - Custom Rocoto build location
  • TODO placeholders for actual role account paths per machine

Testing Strategy

  • Multi-platform validation across hera, gaeac6, orion, hercules, ursa, and wcoss2
  • Full pipeline coverage including builds, experiments, and CTests
  • Stress testing with dryrun feature under production CI loads
  • Graceful fallback to system Rocoto if custom path not available

Rollback Plan

These are temporary changes designed for easy removal once upstream Rocoto PR is merged:

  1. Remove the PATH override sections from CI templates
  2. Remove the system-specific settings sections from platform configs

Acceptance Criteria

  • All CI pipelines use configurable Rocoto build path per platform
  • Dryrun feature accessible for stress testing
  • Platform-specific role account path flexibility
  • Changes are easily reversible
  • Graceful fallback to system Rocoto
  • No disruption to existing workflow functionality

Copilot AI review requested due to automatic review settings November 13, 2025 15:11
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA marked this pull request as draft November 13, 2025 15:12
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.


# HPC account which overides the default account
export HPC_ACCOUNT=${HPC_ACCOUNT:-fv3-cpu}
export HPC_ACCOUNT=${HPC_ACCOUNT:-epic-ps}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably want to revert this to fv3-cpu before merging. Fine for testing.

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added the CI/CD Issue related to CI/CD label Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD Issue related to CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update CI Pipelines to use Rocoto feature branch with --dryrun for stress testing

3 participants