Skip to content

Add pipeline analysis next-steps workflow#48154

Merged
ReilleyMilne merged 9 commits into
mainfrom
reilleymilne/pipeline-analysis-workflow
Jul 20, 2026
Merged

Add pipeline analysis next-steps workflow#48154
ReilleyMilne merged 9 commits into
mainfrom
reilleymilne/pipeline-analysis-workflow

Conversation

@ReilleyMilne

Copy link
Copy Markdown
Member

Adds an agentic workflow that comments on a PR when its Azure DevOps CI fails, with a short summary of what broke and suggested next steps. Includes both the trigger and markdown steps.

Copilot AI review requested due to automatic review settings July 20, 2026 17:25
@ReilleyMilne
ReilleyMilne requested a review from a team as a code owner July 20, 2026 17:25
@ReilleyMilne ReilleyMilne self-assigned this Jul 20, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds automated Azure DevOps pipeline-failure analysis and PR guidance.

Changes:

  • Dispatches analysis for failed CI check suites.
  • Generates concise next-step comments using Copilot.
  • Adds the compiled agentic workflow.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
pipeline-analysis-next-steps.md Defines analysis and commenting behavior.
pipeline-analysis-next-steps.lock.yml Compiled executable workflow.
pipeline-analysis-next-steps-trigger.yml Dispatches analysis after CI failures.

Comment thread .github/workflows/pipeline-analysis-next-steps.md Outdated
Comment thread .github/workflows/pipeline-analysis-next-steps.md
Comment thread .github/workflows/pipeline-analysis-next-steps-trigger.yml Outdated
Copilot AI review requested due to automatic review settings July 20, 2026 17:53
Comment thread .github/workflows/pipeline-analysis-next-steps-trigger.yml Outdated
Comment thread .github/workflows/pipeline-analysis-next-steps.md

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

Comment thread .github/workflows/pipeline-analysis-next-steps.md
Comment thread .github/workflows/pipeline-analysis-next-steps.md Outdated
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Copilot AI review requested due to automatic review settings July 20, 2026 18:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

Comment thread .github/workflows/pipeline-analysis-next-steps.md Outdated
Copilot AI review requested due to automatic review settings July 20, 2026 18:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

Comment thread .github/workflows/pipeline-analysis-next-steps-trigger.yml Outdated
Comment thread .github/workflows/pipeline-analysis-next-steps-trigger.yml Outdated
Copilot AI review requested due to automatic review settings July 20, 2026 18:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

.github/workflows/pipeline-analysis-next-steps-trigger.yml:45

  • This condition does not verify the check producer, so any GitHub App whose failed check happens to end in - pullrequest can dispatch this workflow. Existing check-run consumers gate on github.event.check_run.check_suite.app.name == 'Azure Pipelines' (post-apiview.yml:21, verify-links.yml:24); add the same gate here.
      (endsWith(github.event.check_run.name, '- pullrequest') &&
       github.event.check_run.conclusion == 'failure')

Comment thread test-trigger-pipeline.yml Outdated
Comment thread .github/workflows/pipeline-analysis-next-steps-trigger.yml
Copilot AI review requested due to automatic review settings July 20, 2026 18:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

.github/workflows/pipeline-analysis-next-steps-trigger.yml:45

  • This gate is broader than the documented python - pullrequest check: any failed check whose name merely ends with - pullrequest can dispatch the billable agent workflow. Match the exact Azure Pipelines rollup name so unrelated GitHub Apps or Actions jobs cannot trigger analysis.
    if: >
      github.event_name != 'check_run' ||
      (endsWith(github.event.check_run.name, '- pullrequest') &&
       github.event.check_run.conclusion == 'failure')

.github/workflows/pipeline-analysis-next-steps-trigger.yml:101

  • Passing a pull-request aw_context activates the generated Checkout PR branch step (lock file lines 487-500), exposing PR-controlled eng/ content to the agent despite the workflow's guarantee that no PR code is checked out. The explicit pr_number input and add-comment.target already identify the PR, so omit this context.
              -f aw_context="$aw_context" \

Copilot AI review requested due to automatic review settings July 20, 2026 19:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

.github/workflows/pipeline-analysis-next-steps-trigger.yml:45

  • This condition does not verify that the check was created by Azure Pipelines, so any failing check whose name happens to end in - pullrequest will consume a Copilot run and may post an unrelated analysis. Gate on check_suite.app.name == 'Azure Pipelines' as the existing check-run workflow does in .github/workflows/post-apiview.yml:21.
      (endsWith(github.event.check_run.name, '- pullrequest') &&
       github.event.check_run.conclusion == 'failure')

.github/workflows/pipeline-analysis-next-steps-trigger.yml:101

  • Passing a pull_request aw_context activates gh-aw's generated Checkout PR branch step (pipeline-analysis-next-steps.lock.yml:487-500). That replaces the base eng checkout with PR-controlled content before the agent runs, contradicting this workflow's explicit no-PR-checkout boundary. The explicit pr_number already supplies the prompt and safe-output target, so omit aw_context and recompile the lock file.
              -f aw_context="$aw_context" \

Copilot AI review requested due to automatic review settings July 20, 2026 20:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

.github/workflows/pipeline-analysis-next-steps-trigger.yml:94

  • Passing this PR context activates gh-aw's generated Checkout PR branch step after pipeline-analysis.txt has already been created (lock file lines 474-500). That checkout is unnecessary for this read-only workflow and can fail whenever the target PR adds a root pipeline-analysis.txt, because Git refuses to overwrite the untracked analysis file. The PR number is already supplied explicitly for GitHub reads and the safe-output target, so omit aw_context to keep the agent on the base sparse checkout.
              -f aw_context="$aw_context" \

@ReilleyMilne
ReilleyMilne enabled auto-merge (squash) July 20, 2026 20:33
@ReilleyMilne
ReilleyMilne merged commit cbc3f7f into main Jul 20, 2026
18 checks passed
@ReilleyMilne
ReilleyMilne deleted the reilleymilne/pipeline-analysis-workflow branch July 20, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants