cli-freshness.yml

name: agent-CLI freshness

# "Are the baked agent-CLI pins drifting behind npm?", asked out of band
# (Factory#459).
#
# The three CLIs are pinned in each service's Dockerfile and baked into the
# runtime image (TFactory#791). NOTHING proposes bumps for them: the Renovate
# GitHub App is not installed on this account — zero Renovate PRs across all five
# repos, ever — and Dependabot's docker ecosystem parses only `FROM` lines, so it
# cannot see an `npm install -g @pkg@version` bake step. factory-gitops'
# renovate.json records both facts, and its own customManager has matched nothing
# since the pins moved out of the manifests.
#
# WHAT THIS BUYS, and it is the reason it exists rather than waiting for the App
# to be installed: "no bump PRs" and "nothing to update" look identical from
# outside. That is how the same gap sat unnoticed for nine weeks in Factory#436.
# A freshness assertion cannot be silently not-running — if it stops, this job
# goes red. Installing the Renovate App (Factory#459 option 1) is the fuller fix
# and needs account-level authorisation; the two are complementary.
#
# WHY SCHEDULED AND NOT A PR GATE, which is the OPPOSITE call from Factory#552
# and deliberately so. A pull request does not make a pin stale; TIME does. A
# pre-merge test would fail PRs for something the PR did not cause, which is the
# cry-wolf shape that gets a gate muted (Factory#538). The comparator itself IS
# covered on every hub PR, offline, by tests/test_cli_freshness.py.
#
# IT REPORTS, AND SINCE Factory#459 IT ALSO PROPOSES — but it never merges. A
# broken upstream CLI release must not reach the fleet because a bot merged it
# unattended, so the `propose` job below opens a pull request in the service repo
# and stops there. See scripts/check_cli_freshness.py for why that is a workflow
# rather than Renovate, and for the measurement showing Dependabot cannot see
# these pins even though it bumps the FROM line in the same Dockerfile daily.
#
# WHY THE REPORT NEEDS NO PAT: the service repos are public and the npm registry
# is anonymous. The `propose` job needs one, because opening a pull request is a
# write — and because a pull request opened with GITHUB_TOKEN does not trigger
# workflow runs, which would leave the bump unverified by the very image build
# that makes proposing it safe. That is the objection Factory#459 raised against
# its option 2, and a PAT is what answers it.

on:
  schedule:
    # Weekly, Monday 06:41. These CLIs ship every few days and the window is 30
    # days, so a daily run would repeat the same verdict six times before it
    # could change. Offset from the daily watchdogs (pin-freshness 05:11,
    # chart-vs-gitops 05:41, branch-protection-drift 06:17). Note BST != UTC.
    - cron: "41 6 * * 1"
  workflow_dispatch:
    inputs:
      max_age_days:
        description: "How long a newer release may sit unadopted before this alerts"
        type: string
        required: false
        default: "30"

permissions:
  contents: read

concurrency:
  group: cli-freshness
  cancel-in-progress: false

jobs:
  freshness:
    name: baked agent-CLI pins are not far behind npm
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

      - uses: ./.github/actions/setup-python-311

      - name: Self-test the comparator before trusting it
        # First, not last, for the same reason every other watchdog here does it:
        # a broken comparator cannot be trusted to say "everything is current",
        # so it fails as a broken comparator rather than quietly passing
        # everything (rule 4.7).
        run: python3 scripts/check_cli_freshness.py --self-test

      - name: Compare every baked pin against the npm registry
        # Exit 1 = a pin has been behind for longer than the window. Exit 2 = a
        # Dockerfile or registry entry could not be read, which is also a
        # failure: a watchdog that read nothing must not report freshness.
        env:
          WINDOW: ${{ inputs.max_age_days || '30' }}
        run: |
          set -o pipefail
          python3 scripts/check_cli_freshness.py --max-age-days "${WINDOW}" \
            | tee -a "$GITHUB_STEP_SUMMARY"

      - name: What to do when this job is red
        if: failure()
        run: |
          echo "A pinned agent CLI has been behind npm for longer than the window."
          echo
          echo "SINCE Factory#459 THIS RED MEANS SOMETHING SHARPER than it used to. The"
          echo "propose job below has been opening a bump PR on branch chore/agent-cli-pins"
          echo "in that service repo every week, so a pin this far behind means the PR was"
          echo "opened and left. Go and look at it: <repo>/pulls?q=head:chore/agent-cli-pins"
          echo
          echo "Merge it once its image-build check is green, or close it. Do NOT automate"
          echo "the merge: a broken upstream release reaching the fleet is exactly what"
          echo "that build and the manual pin exist to prevent."
          echo
          echo "If the pin is deliberately held back, raise --max-age-days with the"
          echo "reason rather than muting the job — and say so where the pin lives."
          echo
          echo "NO TRACKED PIN FOUND means the bake step moved again, as TFactory#791"
          echo "moved it out of the gitops manifests. The watchdog is then looking in"
          echo "the wrong place and is checking nothing; fix the path, do not skip."

  propose:
    name: open or refresh the bump PR in each service repo
    runs-on: ubuntu-latest
    timeout-minutes: 30
    # Runs whatever the report said. Being behind is not yet an alert (that is
    # what the 30-day window is for) but it IS already something to propose --
    # which is the whole point: the PR should be sitting there ready long before
    # the alert fires, so that when it fires it means the PR was ignored.
    needs: freshness
    if: always() && needs.freshness.result != 'cancelled'
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

      - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
        with:
          python-version: "3.11"

      - name: Open or refresh one bump PR per service repo
        # Never merges and never enables auto-merge. What makes proposing these
        # automatically defensible is that each service repo's image-build.yml
        # compiles `target: runtime` on every PR and the bake step runs
        # `claude --version` inside that build, so a yanked or broken release
        # fails on the PR. That gate has to actually RUN, which is why this uses
        # a PAT: a PR opened with GITHUB_TOKEN triggers no workflows.
        #
        # GITOPS_PAT is reused rather than minting a fourth fleet credential; it
        # needs contents:write and pull-requests:write on AIFactory, PFactory and
        # TFactory. If it has less, this job fails naming what is missing instead
        # of reporting that it found nothing to do (rule 4.7) -- the same shape
        # branch-protection-drift.yml uses for its own token.
        env:
          GH_TOKEN: ${{ secrets.GITOPS_PAT }}
        run: |
          set -o pipefail
          python3 scripts/check_cli_freshness.py --open-bump-pr \
            | tee -a "$GITHUB_STEP_SUMMARY"