Skip to content

chore(contributors): automated audit script + weekly GitHub Action#621

Merged
yodakanohoshi merged 2 commits into
mainfrom
chore/contributors-audit-automation
Jun 10, 2026
Merged

chore(contributors): automated audit script + weekly GitHub Action#621
yodakanohoshi merged 2 commits into
mainfrom
chore/contributors-audit-automation

Conversation

@masukai

@masukai masukai commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes the recognition-gap detection loop that surfaced 3 distinct forgotten action items during the #531 / #619 / #620 conversation. After this lands we get a weekly tracking issue when .all-contributorsrc falls out of sync, instead of relying on someone noticing in the middle of an unrelated thread.

What landed

File Purpose
scripts/check_contributors.sh Three audit checks (URL drift / missing entries / overdue Triage invitations) — exit 0 clean / 1 mismatch / 2 script error
Makefile (+ check-contributors target) Local sibling to check-i18n / check-changelog / check-skills
.github/workflows/contributors-audit.yml Weekly Monday 00:30 UTC run — opens / updates a tracking issue when a gap is detected

The three checks

# Check Failure mode it would have caught
1 Profile URL drift between README.md and .all-contributorsrc #616#617#620 (Muawiya YouTube URL still in source-of-truth after README was hand-fixed)
2 Contributors with merged PRs missing from .all-contributorsrc @yodakanohoshi (co-Owner!) missing through 4 merged PRs and the v0.7.8 cut — discovered manually only during the #531 conversation. Also flags @Photon101 (9 PRs!) and @kiwamizamurai who #620 missed initially
3 Triage Collaborator eligibility per GOVERNANCE.md (5+ merged PRs, active last 30 days) Would have given a definitive answer to "wait, is Muawiya not Triage Collaborator yet?" — instead of having to grep collaborator lists manually

Bot filtering

Check 2 filters out *[bot], dependabot, renovate, and github-actions accounts so they're not flagged as "missing contributors".

Sample output (against current main before #620 merges)

=== Check 1: profile URL drift (.all-contributorsrc vs README.md) ===
  ✗ @Muawiya-contact: drift
      .all-contributorsrc: https://www.youtube.com/@Coding_Moves
      README.md:        https://muawiya-contact.github.io/muawiya-portfolio/

=== Check 2: contributors with merged PRs missing from .all-contributorsrc ===
  ✗ missing entries (these accounts have merged PRs but no entry):
      @Deadpool2000: 1 merged PRs
      @Godzilaa: 1 merged PRs
      @Photon101: 9 merged PRs
      @kiwamizamurai: 2 merged PRs
      @nvphungdev: 1 merged PRs
      @safridwirizky: 1 merged PRs
      @sap6011: 1 merged PRs
      @yodakanohoshi: 4 merged PRs

=== Check 3: GOVERNANCE.md Triage Collaborator eligibility (5+ merged PRs, active last 30 days) ===
  ✓ no overdue Triage Collaborator invitations

After #620 merges, Check 1 + the @yodakanohoshi line of Check 2 will go away. I'll add @Photon101 and @kiwamizamurai to #620 before it merges (real misses the script just surfaced). The remaining 1-PR drive-by contributors will get a separate metadata pass.

Workflow behaviour

The weekly Action runs bash scripts/check_contributors.sh. On non-zero exit:

  • If a tracking issue titled "chore(contributors): weekly audit found gaps in .all-contributorsrc" is already open, the workflow appends a comment with the new report (so multiple weeks of drift don't fragment).
  • Otherwise, it opens a fresh tracking issue with the report.

This matches the pattern used by check-changelog-required (#590) — surface gaps as actionable signals, never silently accumulate.

Out of scope (intentionally deferred)

  • Phase 2c — PR-label → all-contributors badge auto-suggest. Depends on whether the @all-contributors GitHub App is installed on the repo and on the bot accepting webhook-invoked commands. Needs a separate check before wiring.
  • Phase 3 (Monthly Contributor Spotlight policy + Owner promotion criteria automation): policy decisions, needs co-Owner discussion via GOVERNANCE.md process.

Test plan

  • bash scripts/check_contributors.sh locally — output above
  • make check-contributors — same exit code
  • Bot filtering verified (allcontributors[bot] / dependabot[bot] no longer in "missing" list)
  • CI green on this PR
  • Workflow dispatched manually after merge to validate the issue-opening path

[skip changelog] — tooling-only addition, no user-facing or release-note impact.

🤖 Generated with Claude Code

Closes the recognition gap detection loop that surfaced 3 distinct
forgotten action items during the #531 / #619 / #620 conversation:

  1. URL drift between README.md and .all-contributorsrc — what
     caused #616#617#620 (Muawiya YouTube URL).
  2. Contributors with merged PRs entirely missing from
     .all-contributorsrc — what hit @yodakanohoshi, @Photon101,
     @kiwamizamurai before #620.
  3. Triage Collaborator invitations overdue per GOVERNANCE.md
     criteria — what we briefly thought was the case for Muawiya
     (incorrectly, since they were already invited in #363).

## What landed

- `scripts/check_contributors.sh` — runs the three checks locally
  or in CI. Exit codes: 0 clean / 1 mismatch / 2 script error.
  Filters bot accounts (allcontributors[bot], dependabot[bot],
  renovate, github-actions) from "missing" detection. Test 3 cross-
  references the repo collaborators list to skip accounts already
  granted triage permission.

- `make check-contributors` — Makefile target alongside the existing
  `check-i18n` / `check-changelog` / `check-skills` family.

- `.github/workflows/contributors-audit.yml` — weekly run (Monday
  00:30 UTC, offset from the existing nightly / weekly schedules to
  avoid queue contention). On non-zero exit, opens or updates a
  tracking issue with the full report so the gap is acted on rather
  than silently accumulating.

## Verified

Running the script against current main (before the Phase 1 audit
PR #620 has merged) correctly surfaces:

  - URL drift on @Muawiya-contact (will go away once #620 merges)
  - 7 missing entries beyond @yodakanohoshi: @Photon101 (9 PRs!),
    @kiwamizamurai, @Deadpool2000, @Godzilaa, @nvphungdev,
    @safridwirizky, @sap6011

The @Photon101 and @kiwamizamurai misses will be added to PR #620
before that PR merges; the rest (1-PR drive-by contributors) follow
as a separate metadata pass.

## Out of scope

- `Phase 2c`: PR-label → all-contributors badge auto-suggest. That
  one depends on the @all-contributors GitHub App being installed on
  the repo and the bot accepting bot-invoked commands — needs a
  separate check before wiring.

- `Phase 3` (Monthly Spotlight policy + Owner promotion criteria
  automation): policy decisions, needs co-Owner discussion.

[skip changelog] — tooling-only addition, no user-facing or
release-note impact.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

masukai added a commit that referenced this pull request Jun 7, 2026
Discovered by scripts/check_contributors.sh (Phase 2, PR #621):

- @Photon101 — 9 merged PRs (badges: code, maintenance)
- @kiwamizamurai — 2 merged PRs (badges: code, bug)

Both qualify as Triage Collaborator candidates under
GOVERNANCE.md (5+ PRs OR major contribution end-to-end);
Photon101 in particular has been steadily active over the
last 30 days.

Also re-syncs README.ja.md i18n marker against the new
README.md base hash.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@masukai masukai requested a review from yodakanohoshi June 7, 2026 22:29
@masukai

masukai commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

@yodakanohoshi — small ping when you have a window 🙏

Scripts + Makefile target + weekly GitHub Action that audits .all-contributorsrc for drift, missing entries, and Triage Collaborator eligibility. CI green, the script was the one that surfaced @Photon101 / @kiwamizamurai as missing on #620.

No rush — just flagging.

@yodakanohoshi

Copy link
Copy Markdown
Contributor

Nice — this closes the recognition-gap loop cleanly. The 0/1/2 exit-code contract and the bot filtering both look right, and it already proved itself surfacing @Photon101 / @kiwamizamurai on #620.

One thing to sanity-check: the create path uses --label "chore,contributors", but I don't see a contributors label in the repo or in .github/labels.yml (didn't verify exhaustively). If it's genuinely absent, gh issue create would fail on the first run that opens an issue — the comment-update branch is fine. Probably cleanest to add it to labels.yml so sync-labels owns it rather than a manual gh label create. Want me to fold that in?

Otherwise good to go — happy to merge once the label question is settled.

masukai added a commit that referenced this pull request Jun 10, 2026
…hoshi, badge expansion) (#620)

* chore(contributors): evidence-based all-contributors audit (+yodakanohoshi, badge expansion)

Audit of `.all-contributorsrc` and the README contributor table against
actual observable contributions. Every badge change here has explicit
PR/issue evidence — no role-based defaults, no self-promotion.

## What changed

### Missing entry added

- **@yodakanohoshi** — co-Owner per GOVERNANCE.md, listed in repo collaborators,
  but was entirely absent from `.all-contributorsrc`. Added with badges
  reflecting actual contributions:
  - code: #610 ClickHouse identifier fix (substantial)
  - doc: #466 i18n marker sync
  - maintenance: #440 post-merge cleanup, #513 CHANGELOG tidy
  - ideas: 10 reported issues (#258 staged_upload, #413 drt diff,
    #414 sync failure alerts, #415 field_mappings, #416 Amplitude,
    #417 Mixpanel, #418-421 destination requests)
  - review: PR reviews as Owner (e.g. #613 S3)
  - projectManagement: co-Owner role per GOVERNANCE.md

### URL fix (source-of-truth sync)

- **@Muawiya-contact** — `.all-contributorsrc` still pointed to the old
  YouTube URL (`https://www.youtube.com/@Coding_Moves`). The README was
  fixed in #616 but the source-of-truth file wasn't, so the next
  `all-contributors generate` would have reverted the README change.
  Synced to the correct portfolio URL (`https://muawiya-contact.github.io/muawiya-portfolio/`).

### Badge expansions (evidence-based)

- **@masukai** (code → code, doc, maintenance, ideas, tool, projectManagement,
  review, mentoring):
  - doc: connector docs #614, mirror docs #607, ROADMAP, CLAUDE.md, release notes
  - maintenance: release cuts (#602, #611), CHANGELOG monotonic check
  - ideas: v0.7.x narrative design, #619 Phase 3 sub-issue, milestone planning
  - tool: `.claude/commands/drt-release-check.md`, drt-create-sync skill
  - projectManagement: milestone management (v0.7.x / v0.8 / v0.9 / v1.0)
  - review: PR #608 (Mixpanel), #610 (ClickHouse), #615 (OTel Phase 2)
  - mentoring: #619 honest-feedback reply, #584 BigQuery coaching

- **@Muawiya-contact** (code → code, doc, infra, maintenance, ideas):
  - doc: OPEN_CORE.md #456, VERSIONING.md #457 #464 — "writing the rules
    of this project" contributions that most contributors never touch
  - infra: all-contributors workflow #436, OTel CI integration
  - maintenance: #492 orphan shadow cleanup, #478 deprecation warnings
  - ideas: #616 CI env vuln flag (cross-PR awareness), Phase 3 wireup

- **@cian-ps** (code → code, bug, infra):
  - code preserved: #529 (`drt init` "Next steps"), #574 (Amplitude destination)
  - bug added: #561 CI mypy mismatch report
  - infra added: #323 FUNDING.yml + Sponsor badge

### Badge corrections (mis-tagged code → actual contribution type)

- **@xtreellaDev** (code → test): #325 was a contract test PR
- **@Ai-chan-0411** (code → doc): #313 was a docs-only PR (connector
  support matrix with auth methods)

### Unchanged (correctly tagged already)

@Khush-domadia, @Pawansingh3889, @PFCAaron12, @armorbreak001, @pureqin,
@wahajahmed010, @GokulKashyap — all correctly tagged based on their
PR history.

## Methodology

Followed all-contributors evidence-based principles:
- Badges reflect observable contributions, not titles
- Every badge has cited PR / issue evidence in this commit message
- Owner badges receive the same evidence requirement as anyone else
  (no "Owner therefore many badges" shortcut)
- Order in `.all-contributorsrc` places Owners first
  (masukai, yodakanohoshi), then by initial contribution chronology

README badge count: 12 → 13. README + README.ja contributor tables
regenerated to match.

[skip changelog] — recognition / metadata only, no behaviour change.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(contributors): add @Photon101 and @kiwamizamurai

Discovered by scripts/check_contributors.sh (Phase 2, PR #621):

- @Photon101 — 9 merged PRs (badges: code, maintenance)
- @kiwamizamurai — 2 merged PRs (badges: code, bug)

Both qualify as Triage Collaborator candidates under
GOVERNANCE.md (5+ PRs OR major contribution end-to-end);
Photon101 in particular has been steadily active over the
last 30 days.

Also re-syncs README.ja.md i18n marker against the new
README.md base hash.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(readme): bump all_contributors badge 13 → 15 to match the 15-entry table

yodakanohoshi caught this in #620 review: README.md's badge still
read `all_contributors-13` while its own contributor table lists 15
people and README.ja.md was already correct at 15. The 13 was a
stale intermediate value from when this PR added only yodakanohoshi
(12 → 13) before @Photon101 / @kiwamizamurai were added (→ 15);
README.ja.md got the 15 bump but README.md was missed.

Now consistent: badge (both languages) + table (both languages) +
.all-contributorsrc all read 15.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(i18n): bump README.ja marker to the badge-fix commit

Keeps `make check-i18n` clean after the badge correction — the marker
was already one commit stale on this branch (recorded 79b6171, base
updated at 3776c6c), and the badge fix moved README.md's tip again.
Content is identical between the two READMEs; this is just the marker
hash.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…n the audit workflow

yodakanohoshi caught this in #621 review: the issue-create path uses
`--label "chore,contributors"`, but only `chore` existed in the repo
/ .github/labels.yml — `contributors` was absent, so the very first
audit run that opens an issue would hard-fail on `gh issue create`
(the comment-update branch was fine; only the create branch was at
risk).

Two-part fix:

1. **`.github/labels.yml`** — add `contributors` (new "Meta" section)
   so `sync-labels` owns it as the source of truth, per yodakanohoshi's
   suggestion. Colour 5319e7 (governance purple), description ties it
   to the recognition/governance automation.

2. **workflow self-bootstrap** — add an idempotent
   `gh label create contributors --force || true` immediately before
   the `gh issue create` in the open-new-issue branch. This removes
   the dependency on `make sync-labels` having run first: the audit
   workflow can open its tracking issue on a fresh repo state without
   a missing-label failure, regardless of label-sync ordering.
   `--force` is update-or-create (idempotent); labels.yml remains the
   SSoT and sync-labels reconciles colour/description on its own runs.

The duplicated colour/description in the workflow guard is
intentional and cosmetic — sync-labels overwrites it from labels.yml.
Keeping the guard self-contained matters more than DRY here because
the whole point is that the workflow must not depend on external
ordering to succeed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@masukai

masukai commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Great catch — you're right, contributors didn't exist (only chore did), so the first run that opens an issue would've hard-failed on gh issue create. Fixed in two parts:

1. .github/labels.yml — added contributors under a new "Meta" section so sync-labels owns it as the source of truth, exactly as you suggested. Colour 5319e7 (governance purple), description tied to the recognition/governance automation.

2. workflow self-bootstrap — added an idempotent gh label create contributors --force || true right before the gh issue create in the open-new-issue branch. This makes the workflow self-sufficient: it can open its tracking issue even on a fresh repo state, without depending on make sync-labels having run first. --force is update-or-create, and labels.yml stays the SSoT (sync-labels reconciles colour/description on its own runs).

I went with both rather than just labels.yml because of the bootstrap-ordering window — labels.yml is declarative, so the label only materialises when sync-labels runs, and the audit could fire (weekly cron / push to main) before that. The self-bootstrap guard closes that window so a missing label can never hard-fail the issue creation. The duplicated colour/description in the guard is cosmetic — sync-labels overwrites it from labels.yml.

Thanks for not letting that slip through — it's exactly the kind of first-run failure that's invisible until it bites. Ready for another look. 🙏

masukai added a commit that referenced this pull request Jun 10, 2026
…self-bootstrap guard

Self-review before merge caught a latent label time-bomb — the same
class of first-run failure yodakanohoshi flagged on #621's
contributors-audit workflow.

The drift-check workflow opened its tracking issue with
`--label "documentation"`. That label exists in the repo *right now*
only as a leftover GitHub default — `.github/labels.yml` defines
`docs`, and `sync-labels` runs `crazy-max/ghaction-github-labeler`
with `skip-delete: false`. So the moment sync-labels reconciles
(which #621 will trigger by adding the `contributors` label),
`documentation` gets DELETED and `docs` created — at which point
`gh issue create --label "documentation"` would hard-fail on the
next drift run.

Fix mirrors the #621 pattern:
- Use `docs` (the labels.yml canonical) instead of `documentation`.
- Add an idempotent `gh label create docs --force || true` guard
  right before `gh issue create`, so the workflow is self-sufficient
  regardless of whether sync-labels has run yet. labels.yml stays
  the source of truth; the guard's colour/description are cosmetic
  (sync-labels reconciles them).

Verified: YAML valid, `make check-drift` still exits 0, open/update/
close paths all search the same title substring.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@yodakanohoshi yodakanohoshi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The self-bootstrap guard is the better call — closing the labels.yml-vs-cron ordering window so a fresh-state run can't hard-fail on gh issue create is exactly the right move, and keeping labels.yml as the SSoT while --force handles the bootstrap is clean. Verified contributors is now in labels.yml and the guard sits ahead of the create. CI green.

Approving. Thanks for the fast turnaround 🙏

@yodakanohoshi yodakanohoshi merged commit aa5b4c9 into main Jun 10, 2026
8 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 10, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants