Reduce CI cost #19919
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| name: CI Run Decision | ||
|
|
||
| # Single source of truth for "should this commit force-run all CI jobs | ||
| # regardless of path filter?". Used by per-job ``if:`` gates in pull.yml | ||
| # and trunk.yml so the sampling logic isn't repeated per job. | ||
| # | ||
| # Returns ``is-full-run = 'true'`` for: | ||
| # - workflow_dispatch (manual run) | ||
| # - ciflow/* tag pushes (maintainer-forced full run) | ||
| # - push events whose SHA is in the deterministic 25% sample | ||
| # (last hex char in {0,4,8,c}) | ||
| # | ||
| # Returns ``is-full-run = 'false'`` for: | ||
| # - pull_request / pull_request_target (use path filter instead) | ||
| # - push events not matching any of the above (path-filtered runs) | ||
| # | ||
| # See ``viable-strict-gate.yml``: viable/strict only advances on | ||
| # commits where this is true, so the path-filtered fast path doesn't | ||
| # silently advance partial signal. | ||
|
|
||
| on: | ||
| workflow_call: | ||
| outputs: | ||
| is-full-run: | ||
| description: "'true' if this commit should run all CI jobs regardless of path filter; 'false' otherwise." | ||
| value: ${{ jobs.decide.outputs.is-full-run }} | ||
|
|
||
| jobs: | ||
| decide: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| is-full-run: ${{ steps.compute.outputs.is-full-run }} | ||
| steps: | ||
| - name: Compute is-full-run | ||
| id: compute | ||
| env: | ||
| EVENT_NAME: ${{ github.event_name }} | ||
| REF: ${{ github.ref }} | ||
| SHA: ${{ github.sha }} | ||
| run: | | ||
| set -eu | ||
|
|
||
| IS_FULL=false | ||
|
|
||
| case "$EVENT_NAME" in | ||
| workflow_dispatch) | ||
| IS_FULL=true | ||
| ;; | ||
| esac | ||
|
|
||
| case "$REF" in | ||
| refs/tags/ciflow/*) | ||
| IS_FULL=true | ||
| ;; | ||
| esac | ||
|
|
||
| # Deterministic 25% sample on push: SHA's last hex char in {0,4,8,c}. | ||
| # Keep in sync with the sample in viable-strict-gate.yml. | ||
| if [ "$IS_FULL" = "false" ] && [ "$EVENT_NAME" = "push" ]; then | ||
| case "$SHA" in | ||
| *0|*4|*8|*c) IS_FULL=true ;; | ||
| esac | ||
| fi | ||
|
|
||
| echo "Event: $EVENT_NAME" | ||
| echo "Ref: $REF" | ||
| echo "SHA: $SHA" | ||
| echo "is-full-run: $IS_FULL" | ||
| echo "is-full-run=$IS_FULL" >> "$GITHUB_OUTPUT" | ||
Check warningCode scanning / CodeQL Workflow does not contain permissions Medium |
||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| name: Promote commit to viable/strict | ||
|
|
||
| # Manual escape hatch for the sampled-CI gating in | ||
| # `_ci-run-decision.yml` + `viable-strict-gate.yml`. | ||
| # | ||
| # Pushes a `ciflow/trunk/<sha>` tag at a chosen commit, which: | ||
| # 1. Re-triggers `pull.yml` / `trunk.yml` against that commit with | ||
| # ``is-full-run = true`` (every gated job runs regardless of | ||
| # path filter or SHA sample). | ||
| # 2. Triggers `viable-strict-gate.yml` for that commit; the gate | ||
| # succeeds because tag pushes always count as a full-run. | ||
| # | ||
| # Once those tag-triggered runs all pass, the next | ||
| # `update-viablestrict` cron run will be able to advance viable/strict | ||
| # to the chosen commit. | ||
| # | ||
| # Use cases: | ||
| # - Bisecting a regression on a non-sampled commit. | ||
| # - Pre-release validation: pin viable/strict to a specific commit | ||
| # (e.g. release branch tip) regardless of its SHA's sample bit. | ||
| # - Recovering when recent sampled commits all happen to be red. | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| sha: | ||
| description: "Full 40-char SHA on main / release/* to promote" | ||
| required: true | ||
| type: string | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| concurrency: | ||
| # One in-flight promotion at a time; safer than racing tag pushes. | ||
| group: promote-to-viable-strict | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| push-ciflow-tag: | ||
| if: ${{ github.repository_owner == 'pytorch' }} | ||
| runs-on: ubuntu-22.04 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Validate SHA and push ciflow tag | ||
| env: | ||
| SHA: ${{ inputs.sha }} | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| # Reject anything that isn't a full 40-char lowercase hex SHA. | ||
| if [[ ! "$SHA" =~ ^[0-9a-f]{40}$ ]]; then | ||
| echo "::error::Input must be a full 40-char lowercase hex SHA; got: '$SHA'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # The commit must exist locally (fetch-depth: 0 above pulls | ||
| # everything, but defensively confirm it's an object). | ||
| if ! git cat-file -e "$SHA^{commit}" 2>/dev/null; then | ||
| echo "::error::SHA $SHA is not a commit in this repository." | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Restrict promotion to commits reachable from a release-track | ||
| # branch. Prevents tagging arbitrary commits (PR heads, | ||
| # rewritten branches, etc.) that aren't part of the official | ||
| # main/release history. | ||
| REACHABLE=false | ||
| for branch in main $(git branch -r | grep -E 'origin/release/' | sed 's|origin/||'); do | ||
| if git merge-base --is-ancestor "$SHA" "origin/$branch" 2>/dev/null; then | ||
| echo "SHA is reachable from origin/$branch" | ||
| REACHABLE=true | ||
| break | ||
| fi | ||
| done | ||
| if [ "$REACHABLE" = "false" ]; then | ||
| echo "::error::SHA $SHA is not reachable from main or any release/* branch." | ||
| exit 1 | ||
| fi | ||
|
|
||
| TAG="ciflow/trunk/$SHA" | ||
|
|
||
| # If the tag already exists (e.g. someone already promoted | ||
| # this commit), exit cleanly — no-op is a valid outcome. | ||
| if git ls-remote --tags --exit-code origin "refs/tags/$TAG" >/dev/null 2>&1; then | ||
| echo "Tag $TAG already exists on origin; nothing to do." | ||
| exit 0 | ||
| fi | ||
|
|
||
| git config user.name "pytorchbot" | ||
| git config user.email "pytorchbot@users.noreply.github.com" | ||
| git tag "$TAG" "$SHA" | ||
| git push origin "$TAG" | ||
|
|
||
| echo "::notice::Pushed $TAG. Watch the tag-triggered workflow runs (pull / trunk / viable-strict-gate); once they pass, the next update-viablestrict cron (every 30 min) will advance viable/strict." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can hit 100 consecutive sha ending it 0,4,8,c in a row :p
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very unlikely, but I take the point :)
I'll change sampling to be based on git history depth so it regularly runs every 4 commits