Add pipeline analysis next-steps workflow#48154
Conversation
|
Azure Pipelines: Successfully started running 1 pipeline(s). 9 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Adds automated Azure DevOps pipeline-failure analysis and PR guidance.
Changes:
- Dispatches analysis for failed CI check suites.
- Generates concise next-step comments using Copilot.
- Adds the compiled agentic workflow.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
pipeline-analysis-next-steps.md |
Defines analysis and commenting behavior. |
pipeline-analysis-next-steps.lock.yml |
Compiled executable workflow. |
pipeline-analysis-next-steps-trigger.yml |
Dispatches analysis after CI failures. |
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
This reverts commit 12764ed.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
.github/workflows/pipeline-analysis-next-steps-trigger.yml:45
- This condition does not verify the check producer, so any GitHub App whose failed check happens to end in
- pullrequestcan dispatch this workflow. Existing check-run consumers gate ongithub.event.check_run.check_suite.app.name == 'Azure Pipelines'(post-apiview.yml:21,verify-links.yml:24); add the same gate here.
(endsWith(github.event.check_run.name, '- pullrequest') &&
github.event.check_run.conclusion == 'failure')
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
.github/workflows/pipeline-analysis-next-steps-trigger.yml:45
- This gate is broader than the documented
python - pullrequestcheck: any failed check whose name merely ends with- pullrequestcan dispatch the billable agent workflow. Match the exact Azure Pipelines rollup name so unrelated GitHub Apps or Actions jobs cannot trigger analysis.
if: >
github.event_name != 'check_run' ||
(endsWith(github.event.check_run.name, '- pullrequest') &&
github.event.check_run.conclusion == 'failure')
.github/workflows/pipeline-analysis-next-steps-trigger.yml:101
- Passing a pull-request
aw_contextactivates the generatedCheckout PR branchstep (lock file lines 487-500), exposing PR-controlledeng/content to the agent despite the workflow's guarantee that no PR code is checked out. The explicitpr_numberinput andadd-comment.targetalready identify the PR, so omit this context.
-f aw_context="$aw_context" \
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
.github/workflows/pipeline-analysis-next-steps-trigger.yml:45
- This condition does not verify that the check was created by Azure Pipelines, so any failing check whose name happens to end in
- pullrequestwill consume a Copilot run and may post an unrelated analysis. Gate oncheck_suite.app.name == 'Azure Pipelines'as the existing check-run workflow does in.github/workflows/post-apiview.yml:21.
(endsWith(github.event.check_run.name, '- pullrequest') &&
github.event.check_run.conclusion == 'failure')
.github/workflows/pipeline-analysis-next-steps-trigger.yml:101
- Passing a
pull_requestaw_contextactivates gh-aw's generated Checkout PR branch step (pipeline-analysis-next-steps.lock.yml:487-500). That replaces the baseengcheckout with PR-controlled content before the agent runs, contradicting this workflow's explicit no-PR-checkout boundary. The explicitpr_numberalready supplies the prompt and safe-output target, so omitaw_contextand recompile the lock file.
-f aw_context="$aw_context" \
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
.github/workflows/pipeline-analysis-next-steps-trigger.yml:94
- Passing this PR context activates gh-aw's generated
Checkout PR branchstep afterpipeline-analysis.txthas already been created (lock file lines 474-500). That checkout is unnecessary for this read-only workflow and can fail whenever the target PR adds a rootpipeline-analysis.txt, because Git refuses to overwrite the untracked analysis file. The PR number is already supplied explicitly for GitHub reads and the safe-output target, so omitaw_contextto keep the agent on the base sparse checkout.
-f aw_context="$aw_context" \
Adds an agentic workflow that comments on a PR when its Azure DevOps CI fails, with a short summary of what broke and suggested next steps. Includes both the trigger and markdown steps.