Skip to content

fix(ci): external review lost to its own concurrency group, then to OIDC - #58

Merged
yuting0624 merged 2 commits into
masterfrom
fix/external-review-token-and-concurrency
Aug 12, 2026
Merged

fix(ci): external review lost to its own concurrency group, then to OIDC#58
yuting0624 merged 2 commits into
masterfrom
fix/external-review-token-and-concurrency

Conversation

@yuting0624

Copy link
Copy Markdown
Owner

Both found by #57 — the first fork PR to get far enough to hit them. Neither had ever been exercised: every prior run of this workflow was skipped.

1. A run that was going to skip cancelled the one that was going to review

Two labels were applied in the same second, so GitHub sent two labeled events:

run event label authorise
31556100775 (02:11:51) claude-review cancelled
31556102147 (02:11:53) documentation skipped

The documentation run entered the concurrency group with cancel-in-progress: true, killed the live one, and then skipped itself. The PR showed no review and nothing saying why.

This is #42 / #53 in quorum-review, in the one workflow that never got the fix: cancel-in-progress is evaluated before any job condition, so a run that will be skipped still enters the group.

cancel-in-progress: ${{ github.event.action == 'synchronize' }}

A label never makes a running review obsolete — the head commit is unchanged. Only a push does. And false queues rather than discards, so an unrelated label waits its turn and then skips.

2. With that cleared, the action ran for the first time and 401'd

Exchanging OIDC token for app token...
App token exchange failed: 401 Unauthorized - Invalid OIDC token   (x3)

Checked before concluding:

  • Not version skew. The 26 commits between this pin (07-25) and the tag claude-review.yml resolves touch workflow_run support and context.ts, not the exchange.
  • Not a missing App. The internal workflow performs the same exchange and logs App token successfully obtained on every run.
  • The remaining difference is the event: pull_request_target.

Passing github_token explicitly skips the exchange. Upstream documents the input as "optional if using GitHub App" and recommends passing it for the analogous privileged context (workflow_run with a non-write actor), so this is a supported path rather than a workaround.

It is also the better credential here. GITHUB_TOKEN is bounded by this workflow's permissions: block (contents: read, pull-requests: write); an App installation token carries whatever that App was granted across the repository. The narrower one belongs in the pull_request_target job. Cost is cosmetic — comments arrive as github-actions[bot] — plus one benefit: a GITHUB_TOKEN comment does not trigger other workflows, which is the class of failure #42 was.

Tests

260 → 262, each verified by mutation (bare true, an expression that inspects the label, a removed github_token, pull-requests: read).

Written separately from quorum's equivalents rather than shared: quorum discriminates on github.event_name, this file is all pull_request_target and has to key on the action, and an assertion loose enough to pass on both would stop guarding either.

One assertion exists only because the two files differ structurally — this one puts permissions: before concurrency:, so quorum's extraction range comes back empty here and every assertion would have passed on nothing. That is checked explicitly.

Two defects, both surfaced by #57 — the first fork PR to get far enough
to hit them.

1. A skippable run cancelled a live one. Two labels applied in the same
second produce two `labeled` events. The `claude-review` one started
`authorise`; the `documentation` one arrived two seconds later,
cancelled it, and then skipped itself because `github.event.label.name`
was not `claude-review`. Run 31556100775 shows `authorise cancelled`,
31556102147 `authorise skipped`, and the PR showed no review and no
reason. This is #42/#53 in quorum-review, in the one workflow that never
got the fix: `cancel-in-progress` is evaluated BEFORE any job condition.

A label never makes a running review obsolete — the head is unchanged.
Only a push does, and `false` queues rather than discards, so an
unrelated label waits its turn and then skips.

2. With that cleared, the action ran for the first time ever and the
OIDC exchange returned 401 Invalid OIDC token, three attempts. Not a
version skew: the 26 commits between this pin and the tag the internal
workflow resolves touch workflow_run support, not the exchange. The
internal workflow does the same exchange successfully on every run, so
the App is installed and the difference is the event.

Passing github_token explicitly skips the exchange. Upstream documents
the input as "optional if using GitHub App" and recommends passing it
for the analogous privileged context, so this is a supported path, not a
workaround — and it is the better one here. GITHUB_TOKEN is bounded by
this workflow's permissions block; an App installation token carries
whatever that App holds repository-wide. The narrower credential belongs
in the pull_request_target job.

Six assertions, each verified by mutation. The concurrency ones are
written separately from quorum's rather than shared: quorum keys on
github.event_name, this file is all pull_request_target and keys on the
action, and an assertion loose enough for both would stop guarding
either. One of them exists only because the extraction range differs —
this file puts permissions: before concurrency:, so quorum's range comes
back empty here and every assertion would have passed on nothing.

260 -> 262.
@quorum-code-review

quorum-code-review Bot commented Aug 12, 2026

Copy link
Copy Markdown

Quorum review

Reviewing only what changed since a179364. Findings in files this range does not touch are carried over untouched — they were not re-examined, so they are neither re-reported nor treated as fixed.

gemini-3.6-flash, claude-sonnet-5 each read the diff without seeing the other's output (gemini-3.6-flash 0 and claude-sonnet-5 0), which merged to 0 distinct finding(s). 0 of those were reported by both models independently.
The remaining findings were each judged by the model that did not report them: 0 confirmed, 0 uncertain, 0 refuted.

Beyond the diff, the models made 17 read-only lookup(s) into the checkout at 9d30ad7 and opened tests/run-tests.sh, .github/workflows/claude-review-external.yml.

No new issues found in this diff.


Usage
Model Region Calls Input Cached input Output
gemini-3.6-flash global 9 49,355 11,769 184
claude-sonnet-5 global 8 56,605 3,908 7,817

Reviewed 5bbbeb2 · quorum-review 1.7.0 · models gemini-3.6-flash, claude-sonnet-5 · 105s

@claude

claude Bot commented Aug 12, 2026

Copy link
Copy Markdown

Reviewed. This PR only touches .github/workflows/claude-review-external.yml and tests/run-tests.sh — none of the delegate-bash/hook/exit-code surface applies here.

Fix 1 (concurrency): cancel-in-progress: ${{ github.event.action == 'synchronize' }} is evaluated per-run against that run's own event payload, so a labeled run (any label, matching or not) now always queues instead of cancelling, while a synchronize run still preempts a stale review — correctly reproducing the "only a push obsoletes a review" invariant described in the PR body. Traced the #57 scenario by hand: two same-second labeled events both get cancel-in-progress: false now, so the second no longer kills the first mid-run; it queues, then its own authorise: if: correctly no-ops when the label isn't claude-review. Matches the fix already applied to quorum-review.yml.

Fix 2 (github_token): passing github_token: ${{ github.token }} explicitly is well-justified — GITHUB_TOKEN under pull_request_target is bounded by this workflow's own permissions: block (contents: read, pull-requests: write), which is confirmed sufficient to post the review comment. The 401-OIDC claim is backed by cited real run IDs/error text rather than speculation, which is the right bar for an assertion about third-party action behavior.

Tests: the new assertions in run-tests.sh are genuine regression tests, not vacuous ones — I traced each against the pre-fix text: the bare cancel-in-progress: true check, the synchronize-only check, and the github_token presence check would all correctly FAIL against the old file content. The sed -n '/^concurrency:/,/^jobs:/p' and /^permissions:/,/^concurrency:/p extraction ranges are scoped correctly and don't accidentally slurp the large explanatory comment blocks sitting between permissions: and concurrency: in a way that produces a false pass.

No blocking issues found. One purely cosmetic nit: the 401'"'"'d / workflow'"'"'s escaped-quote artifacts in the new comment text (tests/run-tests.sh, around the github_token test) look like leftover heredoc-escaping cruft — harmless since they're inside # comments, but worth a pass if you're touching this section again.

Review nit: '"'"' survived into the comment text as literal characters
when these lines were written through a python heredoc. Harmless — they
are inside # comments — but they read as noise in the one place that
explains why the credential choice matters.
@claude

claude Bot commented Aug 12, 2026

Copy link
Copy Markdown

Reviewed the diff (claude-review-external.yml + tests/run-tests.sh). Note: this PR only touches CI workflow/test files, so the plugin-specific checks (hooks/validate-delegate-bash.sh, exit-code contract, SKILL.md version drift) don't apply here — focusing on the actual change.

Logic itself looks correct. cancel-in-progress: ${{ github.event.action == 'synchronize' }} correctly narrows cancellation to pushes only — since cancel-in-progress is evaluated before any job if:, a labeled event that's about to be skipped will still enter the concurrency group and, under the old bare true, could cancel a live review (exactly the #57 incident described). Under the new expression it just queues, and the job's own if: still filters it out once it dequeues. Traced through both event orderings from the incident and the fix holds.

The new tests are well-built against the exact pitfall this task is primed to look for (a guard/assertion that's a no-op or passes on empty extraction): the XCONC block is explicitly checked for non-emptiness before being used, and the sed range (/^concurrency:/,/^jobs:/p) correctly starts after the long comment block above concurrency: — so the negative assertions (cancel-in-progress: *true, label\.name) can't accidentally match prose in the comments instead of real YAML. Good attention to the ordering quirk (permissions: before concurrency: in this file vs. after in quorum-review.yml).

One thing that doesn't add up: the PR description's test count. It says "260 → 262" (+2), but the diff to tests/run-tests.sh adds 6 independent, unconditionally-executed assertions, not 2:

  1. XCONC non-empty check
  2. cancel-in-progress: *true absence check
  3. github.event.action == 'synchronize' presence check
  4. label\.name|labels\.\* absence check
  5. github_token: ${{ github.token }} presence check
  6. pull-requests: write presence check

Static count of PASS=$((PASS+1)) sites in tests/run-tests.sh goes from 90 (base) to 96 (this branch) — a +6, matching the 6 new blocks, not +2. Worth fixing the description before merge so the number stays trustworthy for the next person who diffs test counts to sanity-check a PR.

Not independently verified (flagging per repo convention rather than assuming): the claim that upstream claude-code-action documents github_token as "optional if using GitHub App" and that omitting it causes the OIDC exchange to 401 under pull_request_target. This sandboxed review session has no outbound network/gh api access, so I couldn't re-check the action's docs or replay the failure. The PR description already does the right thing here — it cites concrete run IDs and the literal 401 Unauthorized - Invalid OIDC token error rather than asserting behavior — so this reads as tested, just noting I couldn't re-verify it myself.

@yuting0624
yuting0624 merged commit 6d0dcb8 into master Aug 12, 2026
6 checks passed
@yuting0624
yuting0624 deleted the fix/external-review-token-and-concurrency branch August 12, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant