Short-circuit duplicate PR validations on draft → ready transitions

[Copilot]

PR validations currently re-run end-to-end when a draft PR is marked ready for review, even if the PR head commit is unchanged and already validated. This adds avoidable queue time and compute usage on identical trees.

• Duplicate-run guard stage

  • Added an EvaluateDuplicate stage at the top of .pipeline/templates/validation-stages.yml.
  • Runs only for PR builds (Build.Reason == PullRequest) in the normal validation path (non-fuzz, non-longhaul).
  • Queries Azure DevOps Build REST API (same System.DefinitionId) and checks for a prior successful run with matching System.PullRequest.SourceCommitId.
  • Uses BUILD_SOURCEBRANCH for the branch-scoped query and supports provider/API trigger fields for matching (pr.sourceSha, pr.sourceCommitId, pr.headSha; plus PR id fields pr.number / pr.pullRequestId).
  • Uses both current PR identifier variables (System.PullRequest.PullRequestId and System.PullRequest.PullRequestNumber) when evaluating candidate runs.
  • Emits output variable skipDuplicate=true|false.

• Stage gating for expensive PR work

  • Updated PR-heavy stages to skip when skipDuplicate == true:
    • Build
    • Kerberos_Test_PR
    • Build_mssql_python
  • Added explicit dependency wiring to consume dependencies.EvaluateDuplicate.outputs['Evaluate.SetDuplicateState.skipDuplicate'] reliably in stage-level conditions.
  • Updated gating to use not(canceled()) so guard failures/skips fall back to full validation instead of suppressing downstream PR stages.

• Guard behavior and safety

  • If token/API lookup is unavailable or fails, guard falls back to skipDuplicate=false (full validation continues).
  • If required PR commit metadata is unavailable, guard also falls back to skipDuplicate=false.
  • Added bounded candidate scan and clear skip log messaging when a prior successful run is found.
  • Added diagnostic logging for duplicate evaluation context, candidate counts, and mismatch/missing-metadata summaries to make draft→ready detection troubleshooting actionable.
  • When the query is branch-scoped, PR identifier mismatches are logged but SHA matching is still allowed; when not branch-scoped, PR identifier mismatch remains a strict filter.

Example of the gating pattern now used:

condition: and(
  not(canceled()),
  eq(variables['Build.Reason'], 'PullRequest'),
  ne(dependencies.EvaluateDuplicate.outputs['Evaluate.SetDuplicateState.skipDuplicate'], 'true')
)

[Copilot]

Pull request overview

Adds a duplicate-run guard to the Azure DevOps validation template so PR validations can skip expensive stages when the PR head commit has already been validated successfully (e.g., draft → ready transitions), reducing redundant CI compute/queue time.

Changes:

• Introduces a new EvaluateDuplicate stage that queries the ADO Build REST API for prior successful PR runs with the same source commit SHA.
• Gates PR-heavy stages (Build, Kerberos_Test_PR, Build_mssql_python) on the guard output (skipDuplicate) and wires stage dependencies to consume the output.
• Adds guard “safe fallback” behavior (intended) by setting skipDuplicate=false when token/API lookup is unavailable.

[David-Engel]

@copilot Address these problems:

1. There are issues reported in the latest PR review.
2. The duplicate evaluation doesn't seem to be working. There was a successful run on a commit, the PR was changed to ready-to-review, and the validation pipeline didn't detect the previous run on the same commit: Pipelines - Runs for mssql-rs Pull request validation

[Copilot]

@copilot Address these problems:

1. There are issues reported in the latest PR review.
2. The duplicate evaluation doesn't seem to be working. There was a successful run on a commit, the PR was change...

Addressed in 2944c06.

I updated .pipeline/templates/validation-stages.yml to:

• fix duplicate lookup matching for draft → ready transitions by querying with BUILD_SOURCEBRANCH and provider-aware triggerInfo SHA/PR ID fields,
• preserve fallback-to-full-validation behavior by replacing succeeded() with not(canceled()) on Build, Kerberos_Test_PR, and Build_mssql_python stage conditions,
• keep guard failures/non-availability on the safe path (skipDuplicate=false) with defensive handling when PR source commit metadata is unavailable.

[David-Engel]

@copilot The duplicate evaluation still isn't working. There was a successful run on a commit, the PR was changed to ready-to-review, and the validation pipeline didn't detect the previous run on the same commit: Pipelines - Runs for mssql-rs Pull request validation
I suggest adding some logs so you can see why the evaluate duplicate isn't working.

[Copilot]

@copilot The duplicate evaluation still isn't working. There was a successful run on a commit, the PR was changed to ready-to-review, and the validation pipeline didn't detect the previous run on the ...

Implemented in 0f8580a.

I added detailed diagnostics to EvaluateDuplicate (context values, candidate count, mismatch/missing-SHA summary, and candidate mismatch logs), and updated PR-id matching to compare both SYSTEM_PULLREQUEST_PULLREQUESTID and SYSTEM_PULLREQUEST_PULLREQUESTNUMBER against trigger fields (pr.number, pr.pullRequestId).

Also, PR-id mismatch is now only enforced when the query is not branch-scoped; when branch-scoped, we log the mismatch and still compare commit SHA so draft → ready transitions on the same head commit can be detected correctly.

[David-Engel]

@copilot The duplicate evaluation still isn't working. There was a successful run on a commit, the PR was changed to ready-to-review, and the validation pipeline didn't detect the previous run on the same commit: Pipelines - Runs for mssql-rs Pull request validation I suggest adding some logs so you can see why the evaluate duplicate isn't working.

@copilot The duplicate skip isn’t working. It looks like the duplicate evaluation is working but stages aren’t being skipped.