Pipelines v3

[ZachGoldberg]

🎈 Gruntwork Pipelines v3.0.0 Release Notes

Pipelines version 3 introduces several new features which include a number of breaking changes. The migration guide below should take less than 5 minutes for most teams to complete, but please do make sure to follow it closely. We're providing a checklist here to help teams ensure that steps are not skipped.

• Install and configure the Gruntwork.io GitHub App
  • (Alternative) Add actions:read permission to PIPELINES_READ_TOKEN
  • (Alternative) (Enterprise Only) Add PR_CREATE_TOKEN to delegated repository workflows
• Update to Terragrunt v0.67.16 or later
• (If necessary) Add new actions to GitHub Organization allow list settings
• (Enterprise Only) Install and activate Drift Detection

Migration Guide

Install and configure the Gruntwork.io GitHub App

Pipelines now uses the Gruntwork.io GitHub App for workflow permissions, which simplifies permission management. Follow the GitHub App Installation Instructions to install the Gruntwork.io GitHub App and add your infrastructure-live-root repository to your account.

Using the Gruntwork.io GitHub App is recommended and provides full feature support for pipelines workflows. Alternatively, if you do not wish to install the Gruntwork.io GitHub App you can modify your existing workflows to continue using GitHub secrets, although some features will not be available. Read more about App only features here.

Alternative steps

1. Add actions: read permissions

Customers explicitly list permissions that Gruntwork Pipelines workflows has by default in the pipelines workflow files in .github/workflows inside their infrastructure repositories. Pipelines v3 now requires actions: read permissions in order to introspect its own runs and provide more helpful logging capabilities.

Customers should make the one-line change to add the actions: read permission in the following files in every repository that uses Gruntwork pipelines (including root, access control and delegated repos). Note, most repositories will have only 1 or two of these workflow files, in which case update what is present and don't worry about the others.

• .github/workflows/pipelines.yml
• .github/workflows/pipelines-drift-detection.yml
• .github/workflows/pipelines-root.yml
• .github/workflows/pipelines-unlock.yml

Old Permissions

permissions:
  id-token: write
  contents: write
  pull-requests: write

New Permissions

permissions:
  id-token: write
  contents: write
  pull-requests: write
  actions: read

2. (Enterprise only) Add PR_CREATE_TOKEN to delegated repository workflows.

Delegated repositories that are not using the Gruntwork.io GitHub App will need to add PR_CREATE_TOKEN: ${{ github.token }} to the secrets in their workflow files:

Old Secrets

secrets:
  PIPELINES_READ_TOKEN: ${{ secrets.PIPELINES_READ_TOKEN }}

New Secrets

secrets:
  PIPELINES_READ_TOKEN: ${{ secrets.PIPELINES_READ_TOKEN }}
  PR_CREATE_TOKEN: ${{ github.token }}

Update to Terragrunt v0.67.16

The latest pipelines works best with the latest version of Terragrunt. Recent versions of Terragrunt have much improved logging, performance and correctness improvements in run-all scenarios with pipelines.

Allowlist Actions

This is only for customers who only allow GitHub actions to run if they are on an explicit allowlist

New actions to add

• gruntwork-io/pipelines-credentials
• gruntwork-io/pipelines-actions/.github/actions/pipelines-drift-detection-consolidate-jobs
• gruntwork-io/pipelines-actions/.github/actions/pipelines-drift-detection-determine-units
• gruntwork-io/pipelines-actions/.github/actions/pipelines-drift-detection-determine-drift
• gruntwork-io/pipelines-actions/.github/actions/pipelines-new-pr-action
• gruntwork-io/pipelines-actions/.github/actions/pipelines-get-job-logs-url

(Enterprise Only) Install and activate Drift Detection

Pipelines Drift Detection can be installed in your repositories by adding a new workflow file:

1. Create a new file at .github/workflows/pipelines-drift-detection.yml
2. Add the following content to the file

name: Pipelines Drift Detection
run-name: "[GWP]: Pipelines Drift Detection"
on:
  # Uncomment to enable scheduled Drift Detection
  # schedule:
  #  - cron: '15 12 * * 1'
  workflow_dispatch:
    inputs:
      path:
        description: (Optional) Path to filter units e.g. "./management/*"
        type: string
      branch-name:
        description: (Optional) branch name to open Drift Detection PRs with
        default: drift-detection
        type: string
permissions:
  id-token: write

jobs:
  GruntworkPipelines:
    uses: gruntwork-io/pipelines-workflows/.github/workflows/pipelines-drift-detection.yml@v3
    with:
      path: ${{ inputs.path }}
      branch-name: ${{ inputs.branch-name }}

New Features

• Pipelines as a GitHub App
• Pipelines Drift Detection (Enterprise Only)
• Improved Account Factory Bootstrap Customization Hooks
• Pipelines Log Link Improvements

[oredavids]

This documentation should be on the runner input. i.e move the path input

[yhakbar]

NIT: This should be refactored out at some point. It's hard how this resolves on first inspection.