ci(triage): issue-triage automation (stale + on-open label/dedup + backfill)

.github/workflows/triage-backfill.yml

@@ -0,0 +1,105 @@
+# .github/workflows/triage-backfill.yml
+# One-time / occasional drain of the ACCRETED backlog — runs the SAME shared
+# analysis (scripts/triage/analyze.js) over EXISTING open issues. Manual only,
+# DRY-RUN by default (preview in the job summary before anything applies), capped.
+name: triage / backfill (manual)
+on:
+  workflow_dispatch:
+    inputs:
+      max_issues:
+        description: "Max issues to process this run (oldest first)"
+        type: number
+        default: 50
+      dry_run:
+        description: "Preview only (no labels/comments applied)"
+        type: boolean
+        default: true
+# Default to no privileges; each job opts in to exactly what it needs.
+permissions: {}
+concurrency:
+  group: triage-backfill
+  cancel-in-progress: false
+jobs:
+  backfill:
+    name: backfill triage over open issues
+    runs-on: ubuntu-latest
+    # Least privilege: read the repo (for analyze.js) + write issue labels/comments.
+    permissions:
+      contents: read
+      issues: write
+    timeout-minutes: 30 # backfill iterates many issues; still bounded
+    steps:
+      - uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
+        with:
+          egress-policy: audit # baseline first — review the run's network report, then flip to "block" with the allow-list below
+          # block-mode allow-list (uncomment + set the model host when flipping):
+          #   allowed-endpoints: >
+          #     github.com:443
+          #     api.github.com:443
+          #     objects.githubusercontent.com:443
+          #     api.anthropic.com:443
+          #     <your-azure-resource>.openai.azure.com:443   # or *.services.ai.azure.com for Foundry
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+        with:
+          persist-credentials: false # only reads scripts/triage/analyze.js; no git ops
+      - uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
+        env:
+          # Model provider — set ONE (Azure preferred so runs draw on shared credits; Anthropic is the fallback).
+          # Non-secret Azure config lives in repo "Variables"; only the API keys are secrets.
+          # zizmor: ignore[secrets-outside-env] read-only inference key; dedicated GH Environment adds friction w/ marginal benefit here
+          AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
+          AZURE_OPENAI_ENDPOINT: ${{ vars.AZURE_OPENAI_ENDPOINT }}
+          AZURE_OPENAI_DEPLOYMENT: ${{ vars.AZURE_OPENAI_DEPLOYMENT }}
+          AZURE_OPENAI_API_VERSION: ${{ vars.AZURE_OPENAI_API_VERSION }}
+          # zizmor: ignore[secrets-outside-env] read-only inference key (fallback provider)
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          # Inputs via env (not inlined into the script) to avoid template-injection.
+          DRY_RUN: ${{ inputs.dry_run }}
+          MAX_ISSUES: ${{ inputs.max_issues }}
+        with:
+          script: |
+            const analyzeIssue = require(`${process.env.GITHUB_WORKSPACE}/scripts/triage/analyze.js`);
+            const { owner, repo } = context.repo;
+            const dryRun = process.env.DRY_RUN === "true";
+            const max = Number(process.env.MAX_ISSUES) || 50;
+            if (!analyzeIssue.providerConfigured()) { core.warning("No model provider configured (set Azure or Anthropic secrets) — nothing to do."); return; }
+            const sleep = (ms) => new Promise(r => setTimeout(r, ms));
+
+            const labels = await github.paginate(github.rest.issues.listLabelsForRepo, { owner, repo, per_page: 100 });
+            const valid = new Set(labels.map(l => l.name));
+            const APPLYABLE = new Set(["bug","enhancement","documentation","question","Integration","SDK","framework","Agent Framework"]); // content labels only; see triage-on-open.yml for rationale
+            const labelList = labels.map(l => `- ${l.name}${l.description ? `: ${l.description}` : ""}`).join("\n");
+
+            const all = await github.paginate(github.rest.issues.listForRepo, { owner, repo, state: "open", sort: "created", direction: "asc", per_page: 100 });
+            const issues = all.filter(i => !i.pull_request).slice(0, max);
+            const rows = [];
+
+            for (const issue of issues) {
+              let r;
+              try { r = await analyzeIssue({ github, owner, repo, issue, labelList }); }
+              catch (e) { core.warning(`#${issue.number}: ${e.message}`); continue; }
+              await sleep(2500); // analyze() does a search — stay under the 30/min search limit
+              if (r.skipped) continue;
+
+              const toApply = r.labels.filter(l => valid.has(l) && APPLYABLE.has(l) && !(issue.labels || []).some(x => (x.name || x) === l));
+              const willLabel = r.labelConfidence >= 0.75 && toApply.length;
+              const dup = (r.duplicateOf && r.dupConfidence >= 0.8 && r.candidates.includes(r.duplicateOf) && valid.has("duplicate")) ? r.duplicateOf : null;
+              if (!willLabel && !dup) continue;
+
+              rows.push({ n: issue.number, labels: willLabel ? toApply.join(",") : "", dup: dup ? `#${dup}` : "" });
+              if (!dryRun) {
+                if (willLabel) await github.rest.issues.addLabels({ owner, repo, issue_number: issue.number, labels: toApply });
+                if (dup) {
+                  await github.rest.issues.addLabels({ owner, repo, issue_number: issue.number, labels: ["duplicate"] });
+                  await github.rest.issues.createComment({ owner, repo, issue_number: issue.number, body: `Possible duplicate of #${dup} — a maintainer will confirm; say so if it's not and we'll remove the label. (Automated triage flag.)` });
+                }
+              }
+            }
+
+            core.summary
+              .addHeading(`Triage backfill — ${dryRun ? "DRY RUN (nothing applied)" : "APPLIED"} · ${issues.length} scanned · ${rows.length} actionable`)
+              .addTable([
+                [{ data: "Issue", header: true }, { data: "Labels", header: true }, { data: "Dup-of", header: true }],
+                ...rows.map(r => [`#${r.n}`, r.labels || "—", r.dup || "—"]),
+              ])
+              .write();

.github/workflows/triage-on-open.yml

@@ -0,0 +1,80 @@
+# .github/workflows/triage-on-open.yml
+# Consolidated on-open triage: label + dedup-flag in ONE combined LLM call
+# (shared logic in scripts/triage/analyze.js). Advisory / high-confidence,
+# reopen-friendly; flags duplicates but never auto-closes. No provider -> clean skip.
+name: triage / on-open
+on:
+  issues:
+    types: [opened]
+# Default to no privileges; each job opts in to exactly what it needs.
+permissions: {}
+concurrency:
+  group: triage-on-open-${{ github.event.issue.number }}
+  cancel-in-progress: true
+jobs:
+  triage:
+    name: label + dedup-flag on open
+    runs-on: ubuntu-latest
+    # Least privilege: read the repo (for analyze.js) + write issue labels/comments.
+    permissions:
+      contents: read
+      issues: write
+    timeout-minutes: 10 # a hung model fetch shouldn't burn the 6h default
+    steps:
+      - uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
+        with:
+          egress-policy: audit # baseline first — review the run's network report, then flip to "block" with the allow-list below
+          # block-mode allow-list (uncomment + set the model host when flipping):
+          #   allowed-endpoints: >
+          #     github.com:443
+          #     api.github.com:443
+          #     objects.githubusercontent.com:443
+          #     api.anthropic.com:443
+          #     <your-azure-resource>.openai.azure.com:443   # or *.services.ai.azure.com for Foundry
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+        with:
+          persist-credentials: false # only reads scripts/triage/analyze.js; no git ops
+      - uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
+        env:
+          # Model provider — set ONE (Azure preferred so runs draw on shared credits; Anthropic is the fallback).
+          # Non-secret Azure config lives in repo "Variables"; only the API keys are secrets.
+          # zizmor: ignore[secrets-outside-env] read-only inference key; dedicated GH Environment adds friction w/ marginal benefit here
+          AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
+          AZURE_OPENAI_ENDPOINT: ${{ vars.AZURE_OPENAI_ENDPOINT }}
+          AZURE_OPENAI_DEPLOYMENT: ${{ vars.AZURE_OPENAI_DEPLOYMENT }}
+          AZURE_OPENAI_API_VERSION: ${{ vars.AZURE_OPENAI_API_VERSION }}
+          # zizmor: ignore[secrets-outside-env] read-only inference key (fallback provider)
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+        with:
+          script: |
+            const analyzeIssue = require(`${process.env.GITHUB_WORKSPACE}/scripts/triage/analyze.js`);
+            const { owner, repo } = context.repo;
+            const issue = context.payload.issue;
+
+            const labels = await github.paginate(github.rest.issues.listLabelsForRepo, { owner, repo, per_page: 100 });
+            const valid = new Set(labels.map(l => l.name));
+            // Allow-list (default-deny): the classifier may apply ONLY these — content
+            // labels that exist in this repo. Excludes disposition/curation labels
+            // (proposal, Roadmap, good first issue, help wanted, release, invalid,
+            // wontfix…) — those are human calls. "enhancement" is this repo's canonical
+            // feature type (heavily used); "Feature Request" is a rarely-used synonym we
+            // deliberately omit to avoid the classifier waffling between the two.
+            const APPLYABLE = new Set(["bug","enhancement","documentation","question","Integration","SDK","framework","Agent Framework"]);
+            const labelList = labels.map(l => `- ${l.name}${l.description ? `: ${l.description}` : ""}`).join("\n");
+
+            const r = await analyzeIssue({ github, owner, repo, issue, labelList });
+            if (r.skipped) { core.info(`skipped: ${r.skipped}`); return; }
+
+            const toApply = r.labels.filter(l => valid.has(l) && APPLYABLE.has(l));
+            if (r.labelConfidence >= 0.75 && toApply.length) {
+              await github.rest.issues.addLabels({ owner, repo, issue_number: issue.number, labels: toApply });
+              core.info(`labeled: ${toApply.join(", ")} (conf ${r.labelConfidence})`);
+            }
+            if (r.duplicateOf && r.dupConfidence >= 0.8 && r.candidates.includes(r.duplicateOf) && valid.has("duplicate")) {
+              await github.rest.issues.addLabels({ owner, repo, issue_number: issue.number, labels: ["duplicate"] });
+              await github.rest.issues.createComment({
+                owner, repo, issue_number: issue.number,
+                body: `Possible duplicate of #${r.duplicateOf} — a maintainer will confirm; if it isn't, just say so and we'll remove the label. (Automated triage flag; nothing closes automatically.)`,
+              });
+              core.info(`flagged possible dup of #${r.duplicateOf} (conf ${r.dupConfidence})`);
+            }

.github/workflows/triage-stale.yml

@@ -0,0 +1,46 @@
+# .github/workflows/triage-stale.yml
+# Stop-the-bleeding MVP #1 — drain the "awaiting info" dead weight.
+# Deterministic (no LLM). Scoped to needs-info only, so it never nukes real issues.
+name: triage / stale
+on:
+  schedule:
+    - cron: "0 9 * * *" # daily 09:00 UTC
+  workflow_dispatch:
+# Default to no privileges; each job opts in to exactly what it needs.
+permissions: {}
+concurrency:
+  group: triage-stale
+  cancel-in-progress: false
+jobs:
+  stale:
+    name: stale needs-info issues
+    runs-on: ubuntu-latest
+    # Least privilege: only label/close stale needs-info issues.
+    permissions:
+      issues: write
+    timeout-minutes: 10
+    steps:
+      - uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
+        with:
+          egress-policy: audit # no secrets/external calls here; GitHub API only. Block-mode: github.com:443 + api.github.com:443
+      - uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9.1.0
+        with:
+          # MVP: only auto-progress issues we explicitly asked info on and got silence.
+          only-labels: "needs-info"
+          days-before-stale: 14
+          days-before-close: 7
+          stale-issue-label: "stale"
+          stale-issue-message: >
+            Marking stale — this has been awaiting the requested info for 14 days
+            and will close in 7 if there's no update. Just comment to keep it open,
+            and reopen anytime if it closes.
+          close-issue-message: >
+            Closing due to inactivity while awaiting more information. Please reopen
+            or comment with the details and we'll pick it back up.
+          remove-stale-when-updated: true
+          exempt-issue-labels: "Roadmap,proposal"
+          # PRs are a later arc — do not touch them here.
+          days-before-pr-stale: -1
+          days-before-pr-close: -1
+          # Be gentle on rate limits.
+          operations-per-run: 60

scripts/triage/README.md

@@ -0,0 +1,72 @@
+# Issue triage automation
+
+Stop-the-bleeding automation for incoming issues. Three workflows, one shared
+analysis module. Everything is **advisory / high-confidence / reopen-friendly** —
+nothing closes an issue on an LLM's say-so.
+
+| Workflow              | Trigger                      | LLM? | What it does                                                                                                                                                                                     |
+| --------------------- | ---------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `triage-stale.yml`    | daily cron                   | no   | Marks `needs-info` issues stale after 14d of silence, closes 7d later. Scoped to `needs-info` only, so it never touches active issues. Exempts `Roadmap,proposal`. PRs untouched. |
+| `triage-on-open.yml`  | `issues: opened`             | yes  | One combined classify + dedup pass. Applies allow-listed labels (conf ≥ 0.75) and flags likely duplicates with an advisory comment (conf ≥ 0.8). Never closes.                                   |
+| `triage-backfill.yml` | manual (`workflow_dispatch`) | yes  | Same analysis over the existing open backlog. **Dry-run by default** — previews in the job summary; only applies when you uncheck dry-run. Capped by `max_issues`.                               |
+
+`analyze.js` is the single source of truth for the LLM logic (search for
+candidates → one combined Anthropic call → return proposals). Both the on-open
+and backfill workflows call it, so policy and safety controls live in one place.
+
+## Setup
+
+1. **Model provider** — the two LLM workflows need ONE provider; with none they
+   clean-skip (log and exit, no failures). `triage-stale` needs nothing.
+   - **Azure OpenAI / Foundry (preferred — draws on shared credits):** secret
+     `AZURE_OPENAI_API_KEY`; repo **Variables** `AZURE_OPENAI_ENDPOINT` and
+     `AZURE_OPENAI_DEPLOYMENT` (optional `AZURE_OPENAI_API_VERSION`, default `2024-10-21`).
+   - **Anthropic (fallback):** secret `ANTHROPIC_API_KEY` (optional `ANTHROPIC_MODEL`).
+   - Force one with the `TRIAGE_PROVIDER` variable (`azure` | `anthropic`); otherwise
+     it's inferred from whichever credentials are present (Azure wins if both are set).
+2. **Curate the label allow-list.** The classifier may apply _only_ the labels in
+   `APPLYABLE` (top of `triage-on-open.yml` / `triage-backfill.yml`). It's
+   default-deny and currently holds the repo's real content labels only —
+   `bug, enhancement, documentation, question, Integration, SDK, framework, Agent Framework`.
+   `enhancement` is this repo's canonical feature type; the rarely-used `Feature Request`
+   synonym is deliberately omitted. Curation/disposition labels (`proposal`, `Roadmap`,
+   `good first issue`, `help wanted`, `release`, `invalid`, `wontfix`…) are excluded —
+   those are human calls. Adjust per repo.
+3. **Create a `needs-info` label** (this repo doesn't have one yet). Until it exists
+   and gets applied, `triage-stale` is a clean no-op — it only acts on `needs-info`.
+4. **First run:** dispatch `triage-backfill` with dry-run **on** and a small
+   `max_issues` to eyeball the proposals before letting it apply anything.
+
+## Safety model
+
+- **Constrained action space** — the LLM never posts free text. It returns
+  structured JSON; the workflow applies validated labels and templated comments.
+- **Default-deny labels** — only allow-listed labels can be applied.
+- **Confidence gates** — labels ≥ 0.75, dedup ≥ 0.8; dedup also requires the
+  target to be one of the candidates we actually searched.
+- **Flag, never close** — duplicates get a label + a "a maintainer will confirm"
+  comment. Humans close.
+- **Spam/low-signal gate** — already-flagged or empty-body-from-outsider issues
+  skip the LLM call entirely (cost guard).
+- **Pinned actions** — checkout / github-script / stale / harden-runner are
+  SHA-pinned, with `persist-credentials: false` (no git ops).
+- **Least privilege** — every workflow defaults to `permissions: {}`; each job
+  opts into only `contents: read` + `issues: write`. Nothing else is granted.
+- **No template injection** — `workflow_dispatch` inputs are passed via `env`
+  and read from `process.env`, never interpolated into the inline script.
+- **Egress monitoring** — [StepSecurity Harden-Runner](https://github.com/step-security/harden-runner)
+  runs first in every job (`egress-policy: audit`). Review the network report,
+  then flip to `block` with the allow-list commented in each workflow so the
+  model key can only reach GitHub + your model host.
+- **Endpoint pinning** — `analyze.js` refuses to send the key to any non-Azure
+  host (guards against the endpoint `var` being repointed).
+- **Bounded runtime** — `timeout-minutes` on every job caps a hung model call.
+
+### Not yet automated (operator actions)
+
+- **Provider spend cap** — set a hard quota/spend limit on the model provider
+  (Azure deployment TPM quota / Anthropic workspace monthly limit). This is the
+  real backstop against issue-flood cost-DoS (an event-driven workflow can't hold
+  a global rate limit); do it when you configure the provider.
+- **Flip Harden-Runner to `block`** after one `audit` baseline run.
+- **Confirm Renovate doesn't auto-merge** action SHA bumps (review them by hand).