diff --git a/.claude/settings.json b/.claude/settings.json index 9cd91ce..e72205e 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -62,6 +62,7 @@ "Skill(sentry-skills:skill-writer)", "Skill(sentry-skills:sred-project-organizer)", "Skill(sentry-skills:sred-work-summary)", + "Skill(sentry-skills:triage-frontend-issues)", "Skill(sentry-skills:typing-exclusion-worker)" ], "deny": [] diff --git a/README.md b/README.md index 87f50ca..e0e6ddf 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,7 @@ Works with Claude Code, Cursor, Cline, GitHub Copilot, and other compatible agen | [skill-writer](skills/skill-writer/SKILL.md) | Canonical workflow to synthesize, create, and iteratively improve agent skills for this repository. | | [sred-project-organizer](skills/sred-project-organizer/SKILL.md) | Take a list of projects and their related documentation, and organize them into the SRED format for submission. | | [sred-work-summary](skills/sred-work-summary/SKILL.md) | Go back through the previous year of work and create a Notion doc that groups relevant links into projects that can then be documented as SRED projects. | +| [triage-frontend-issues](skills/triage-frontend-issues/SKILL.md) | Triage the Sentry `javascript` project queue by archiving non-actionable noise (third-party libs, browser quirks, transient 5xx, test traffic) with `untilEscalating` after user approval. | | [typing-exclusion-worker](skills/typing-exclusion-worker/SKILL.md) | Python typing exclusion worker: remove assigned mypy exclusion modules in small scoped batches, fix typing issues, run validation, and produce a structured completion summary. | ## Available Subagents diff --git a/skills/claude-settings-audit/SKILL.md b/skills/claude-settings-audit/SKILL.md index d1dbdca..6b278b0 100644 --- a/skills/claude-settings-audit/SKILL.md +++ b/skills/claude-settings-audit/SKILL.md @@ -166,6 +166,7 @@ If this is a Sentry project (or sentry-skills plugin is installed), include: "Skill(sentry-skills:skill-writer)", "Skill(sentry-skills:sred-project-organizer)", "Skill(sentry-skills:sred-work-summary)", + "Skill(sentry-skills:triage-frontend-issues)", "Skill(sentry-skills:typing-exclusion-worker)" ] ``` diff --git a/skills/triage-frontend-issues/SKILL.md b/skills/triage-frontend-issues/SKILL.md new file mode 100644 index 0000000..b3e13ac --- /dev/null +++ b/skills/triage-frontend-issues/SKILL.md @@ -0,0 +1,152 @@ +--- +name: triage-frontend-issues +description: Triage new issues in the Sentry `javascript` project by archiving non-actionable noise. Use when asked to "triage issues", "triage the javascript project", "archive non-actionable issues", "triage new frontend issues", or "clean up the sentry/javascript queue". Operates only on the sentry/javascript project, only archives (never resolves), and always archives with `untilEscalating`. +allowed-tools: Read, mcp__sentry__search_issues, mcp__sentry__get_sentry_resource, mcp__sentry__update_issue +--- + +# Triage Frontend Issues + +Archive non-actionable noise from the `sentry/javascript` issue queue: only archive, always `untilEscalating`, always with a stated reason. Issues that look actionable in our code, or that you cannot confidently classify, must be skipped. + +## Hard Rules + +These rules override anything else. Do not relax them. + +1. **Project scope.** Only operate on `organizationSlug=sentry`, project slug `javascript`. If asked to triage a different project, stop and ask the user to confirm. +2. **Archive only.** The only status mutation permitted is `status=ignored`. Never resolve, never unresolve, never assign, never delete, never bulk-update fields other than status. +3. **Always `untilEscalating`.** Use `ignoreMode=untilEscalating`. Never use `forever`, `forDuration`, `untilOccurrenceCount`, or `untilUserCount`. If the user asks for a different mode, stop and have them archive that issue manually — this skill does not perform non-escalating archives. +4. **Always include a `reason`.** The `reason` must be a short, factual sentence naming the category from `references/archive-criteria.md` (e.g., "Third-party library noise — echarts internals; not actionable in our code"). +5. **Never touch issues outside the unresolved queue.** Skip anything with `status` of `resolved`, `ignored`, or `reprocessing`. +6. **Never archive without confirmation.** Build a full plan, show it to the user, wait for explicit approval before calling `update_issue`. A single approval covers the displayed plan only; new batches need new approval. +7. **When in doubt, skip.** If an issue could plausibly be a real bug in our code, do not archive it. Surface it as `needs-human` in the plan with a one-line note. + +## Prerequisites + +- Sentry MCP authenticated via `mcp.sentry.dev`. Required tools: `search_issues`, `get_sentry_resource`, `update_issue`. +- If `update_issue` is not available, stop and ask the user to authenticate the Sentry MCP server. + +## Inputs + +`$ARGUMENTS` is one of: + +| Input shape | Meaning | +|-------------|---------| +| Sentry issue URL (`https://sentry.sentry.io/issues/JAVASCRIPT-…`) | Triage that single issue. | +| Issue short ID (`JAVASCRIPT-…`) | Triage that single issue. | +| Sentry issue query (contains a colon, e.g. `is:unresolved firstSeen:-24h`) | Use as the search query. | +| Empty | Use the default triage queue: `is:unresolved is:unassigned firstSeen:-7d`, sort `new`, limit `50`. | + +If `$ARGUMENTS` is ambiguous, ask the user to clarify before searching. + +## Workflow + +### 1. Load the queue + +For single-issue input: +- Call `get_sentry_resource(url=)` or `get_sentry_resource(resourceType='issue', organizationSlug='sentry', resourceId=)`. +- Confirm `Project` is the javascript frontend project. If not, stop. + +For query/default input: +- Call `search_issues(organizationSlug='sentry', projectSlugOrId='javascript', query=, sort='new', limit=50)`. +- Then call `get_sentry_resource` for each result in parallel to get culprit, substatus, assignee, and stack-frame hints (the search response omits some fields). + +Skip immediately if any of these are true on an issue: + +- `status` is not `unresolved` (already archived, resolved, or in reprocessing). +- `assignedTo` is set to a human (someone is already owning it). +- `assignedTo` is set to a team other than `frontend`/`issues` and the issue looks team-specific (let the owning team triage). + +### 2. Classify each issue + +Read `references/archive-criteria.md` for the category taxonomy with recognition heuristics and examples. For each candidate issue, produce one of: + +| Decision | Meaning | +|----------|---------| +| `archive` | Matches a documented category; include the category name in the reason. | +| `skip` | Could be a real bug in our code, or insufficient evidence; do not archive. | +| `needs-human` | Looks like noise but doesn't cleanly fit a category, or volume is unusually high; flag for user review. | + +When evaluating, weight these signals (in this order): + +1. **Top non-Sentry-SDK frame.** If the top in-app frame is in `node_modules/`, `chrome-extension://`, a third-party host, or ``, this is a strong archive signal. +2. **Title pattern.** Many archives are recognizable from the title alone (see criteria reference). +3. **Volume is not a veto.** Some high-volume issues (10k+ events, thousands of users) are still archive-worthy if the top frame is third-party. Volume alone never forces archive either. +4. **Recency.** Single-event issues older than 30 days with no recurrence are usually noise. +5. **Customer org spread.** If events come from one customer subdomain only (check `customerDomain.subdomain` tag), it is likely customer-environment noise. + +### 3. Build the plan + +Output one Markdown table to the user, in this exact shape: + +``` +## Triage plan — sentry/javascript ( candidates) + +| # | Issue | Title | Volume | Decision | Category | Reason | +|---|-------|-------|--------|----------|----------|--------| +| 1 | [JAVASCRIPT-XXXX](url) | TypeError: ... | 12e/3u | archive | browser-api-noise | Browser clipboard permission denied; not actionable. | +| 2 | [JAVASCRIPT-YYYY](url) | | 4945e/123u | needs-human | — | High volume, no title — please review before archiving. | +| 3 | [JAVASCRIPT-ZZZZ](url) | ZodError: ... | 360e/132u | skip | — | Schema validation failure in our code; looks actionable. | +``` + +Then summarize counts: `N archive / M skip / K needs-human`. End with: + +``` +Reply `apply` to archive the N issues marked `archive`, `apply N,M,...` to archive a subset, or `cancel` to take no action. +``` + +### 4. Apply on approval + +When the user replies `apply` (or `apply `): + +For each issue in the approved set, call: + +``` +update_issue( + organizationSlug='sentry', + issueId=, + status='ignored', + ignoreMode='untilEscalating', + reason=, +) +``` + +Run these sequentially (not in parallel). If a call fails, log the failure, continue with the remaining issues, and report the failed IDs in step 5. + +If the user replies `cancel` or asks to modify the plan, do NOT call `update_issue`. If they reply with edits ("change row 2 to skip"), rebuild the plan and re-confirm. + +### 5. Report + +After applying, output: + +``` +## Triage report + +- Archived: N +- Skipped: M +- Needs human review: K +- Failures: F (with issue IDs) + +
Archived issues + +- JAVASCRIPT-XXXX — +- ... + +
+``` + +## Recovery + +- If `update_issue` fails on one item, log the failure and continue with the rest. Report failed IDs at the end. +- If the user notices a wrong archive, the user can unarchive it themselves in Sentry. The skill never reverses its own actions automatically. +- If the user asks "redo the plan with these tweaks" mid-flow, regenerate the plan from scratch — do not assume the previous plan still applies. + +## Example reasons (use this voice) + +- `Third-party library noise — echarts tooltip; not actionable in our code.` +- `Browser API permission noise — Clipboard writeText denied by user agent.` +- `Customer-environment proxy interference — 200 response treated as error (HTML body from corporate proxy).` +- `Transient backend 5xx — InternalServerError on /api/0/organizations/.../events-meta/; backend transient.` +- `Test/synthetic event — smoke test or security probe, not production traffic.` +- `Wrong project — Prisma/Python error mis-routed to frontend project.` +- `Single-event fluke — 1 event, 1 user, no recurrence in 30+ days.` +- `Browser extension noise — ReferenceError for extension-injected global (DarkReader/WeixinJSBridge).` diff --git a/skills/triage-frontend-issues/SPEC.md b/skills/triage-frontend-issues/SPEC.md new file mode 100644 index 0000000..0938282 --- /dev/null +++ b/skills/triage-frontend-issues/SPEC.md @@ -0,0 +1,66 @@ +# SPEC — triage-frontend-issues + +## Intent + +Reduce noise in the `sentry/javascript` issue queue by archiving issues whose root cause is outside our code: third-party libraries, browser/OS quirks, customer-environment interference, transient backend 5xx, test/synthetic events, and single-event flukes. + +The skill never resolves, never assigns, never deletes — only archives, only with `untilEscalating`, only after explicit user approval. + +## Scope + +- **In scope:** the Sentry `javascript` project under the `sentry` organization. Archive-only triage of unresolved issues. +- **Out of scope:** any other project. Any non-archive status change (resolve, assign, delete, escalate). Any auto-apply mode without confirmation. Any change to issue ownership routing. Anything outside Sentry MCP. + +## Trigger Context + +- User explicitly invokes the skill (`/triage-frontend-issues`) or natural language: "triage the javascript project", "archive non-actionable issues", "clean up the unresolved queue". +- User pastes a `sentry/javascript` issue URL or short ID and asks for a triage decision. + +## Source / Evidence Model + +The category taxonomy in `references/archive-criteria.md` was synthesized from observed historical archive activity in the `sentry/javascript` project. The activity feed for archived issues was fetched and inspected to determine: + +- which `ignoreMode` was used (consistent across the population — drove Hard Rule 3 in `SKILL.md`) +- which kinds of issues are archived versus resolved +- which top-frame and title patterns reliably correspond to non-actionable noise + +Boundary cases were established by inspecting `set_resolved` actions in the same project to identify what the team treats as actionable. Specific counts and per-engineer activity are not captured in this repository. + +## Reference Architecture + +``` +triage-frontend-issues/ +├── SKILL.md # runtime instructions (router) +├── SPEC.md # this file (maintenance contract) +└── references/ + └── archive-criteria.md # category taxonomy with recognition heuristics +``` + +Runtime always reads `SKILL.md`. The criteria reference is loaded during step 2 (classification). SPEC is not loaded at runtime. + +## Evaluation Expectations + +A future iteration of this skill should be evaluated against: + +1. **Precision on positive cases.** Re-run the skill against historical archived issues and confirm it would archive them with a matching category. +2. **Precision on negative cases.** Re-run against historical resolved issues. The skill must mark these `skip`, never `archive`. +3. **Reason quality.** Sampled archive `reason` strings should name a category from the taxonomy and identify a concrete signal (library, API, endpoint). +4. **No-op safety.** With `cancel` reply, no `update_issue` calls are made. + +Hold-out evaluation set should be maintained at `references/evidence/` (not created yet; add when the first false positive is observed). + +## Known Limitations + +- The skill reads issue metadata via Sentry MCP, but the MCP's `get_sentry_resource` output does not surface the activity feed. +- Determining "who last touched this issue" requires HTTP API calls, which the skill does not perform — it defers to the user when an issue's history matters. +- The skill cannot inspect the full stack trace in machine-readable form; it relies on what `get_sentry_resource` returns in its formatted output. +- Edge cases where the top frame is buried may be misclassified — those are intentionally routed to `needs-human`. +- Title pattern matching is heuristic. Novel third-party libraries or browser quirks will not match existing patterns and should be added to `references/archive-criteria.md` after the first occurrence. +- The category taxonomy is sourced from a single project (`sentry/javascript`). Applying it to a different project requires re-validating the categories. + +## Maintenance Notes + +- When the user flags a misclassification, capture the example, decide whether the criteria need updating, and update `references/archive-criteria.md`. +- Re-validate the criteria periodically by sampling recent archives and confirming the patterns still apply. +- If Sentry adds new `ignoreMode` options or changes the `update_issue` API surface, update the Hard Rules and the `update_issue` call snippet in `SKILL.md`. +- Do not expand the skill's scope beyond archiving. Resolution, assignment, and deletion belong to other workflows. diff --git a/skills/triage-frontend-issues/references/archive-criteria.md b/skills/triage-frontend-issues/references/archive-criteria.md new file mode 100644 index 0000000..feb30d2 --- /dev/null +++ b/skills/triage-frontend-issues/references/archive-criteria.md @@ -0,0 +1,151 @@ +# Archive Criteria Taxonomy + +Categories of non-actionable noise in the `sentry/javascript` queue, with recognition heuristics. An issue must match **at least one** category to be archived. Categories are listed in roughly decreasing order of frequency observed in the queue. + +For each category: `Pattern`, `Signals`, `Examples`, `Reason text`. + +--- + +## 1. Single-event fluke + +Issues with exactly one occurrence, one or zero users, that never recurred. + +- **Signals:** `events ≤ 2`, `users ≤ 1`, `firstSeen` and `lastSeen` within ~1 hour of each other, no recurrence in 30+ days. +- **Examples:** isolated `TypeError`s in random anonymous functions, one-off `ReferenceError` from a script that no longer runs. +- **Reason:** `Single-event fluke — N event(s), N user(s), no recurrence.` +- **Caution:** if `firstSeen` is within the last 7 days and the title looks like a real bug ("TypeError" in our app code), prefer `skip` and let it bake. + +## 2. Test / synthetic / security-research + +Events generated by smoke tests, customer security audits, or internal QA probes — not real user traffic. + +- **Signals:** title contains any of: `test`, `smoke test`, `verify test`, `XSS`, `SSRF`, `SSTI`, `CSP test`, `` + - `Error: CSP test ` +- **Reason:** `Test/synthetic event — synthetic traffic from smoke test or security probe.` + +## 3. Wrong-project / non-frontend errors + +Errors that originate in customers' own backends but get reported to `sentry/javascript` because of misconfigured DSNs or shared SDKs. + +- **Signals:** title prefix is `PrismaClientKnownRequestError`, `PrismaClientUnknownRequestError`, `HTTPException`, `AttributeError`, `ImportError`, `ProgrammingError`, or `ZodError` with Python-shaped stacks. +- Culprit looks like a Python module path (e.g., dotted `module.submodule.handler`) or a backend middleware class. +- **Examples:** + - `PrismaClientKnownRequestError:` (any) + - `AttributeError: 'dict' object has no attribute ''` + - `HTTPException` with a Python dotted-module culprit +- **Reason:** `Wrong project — non-frontend error mis-routed (Prisma/Python).` + +## 4. Customer-environment third-party library noise + +Errors whose top in-app frame is in a third-party library that we depend on (or that customers' apps depend on) — not in our code. + +- **Signals:** top frame is under `node_modules/` for a library name like `echarts`, `mobx`, `html2canvas`, `react-dom` (only when the error is React-internal and not our component), `lodash`, etc. +- Also: title references third-party globals injected by browser extensions or wallet providers. +- **Examples:** + - `TypeError: Cannot set properties of null (setting 'innerHTML')` with top frame in `echarts/lib/component/tooltip/...` + - `Error: [MobX] minified error nr: ` (mobx internals) + - `ReferenceError: Can't find variable: DarkReader` + - `TypeError: undefined is not an object (evaluating 'window.ethereum.')` + - `ReferenceError: html2canvas is not defined` +- **Reason:** `Third-party library noise — ; not actionable in our code.` +- **Caution:** if the third-party frame is *reached from* our component code with our state, it might be a misuse on our side. Inspect the second frame down — if it's our `app/` code passing bad input, prefer `skip`. + +## 5. Browser API / permission noise + +Errors caused by browser APIs the user denied, browser quirks, or platform constraints we cannot work around. + +- **Signals:** title contains: + - `NotAllowedError` (clipboard, WebAuthn, permissions, 2FA) + - `OperationError: A request is already pending` (WebAuthn pipeline) + - `SecurityError: Blocked a frame ... cross-origin` + - `SecurityError: Failed to read a named property ... from 'Window'` + - `Failed to execute 'writeText' on 'Clipboard'` + - `WebGL not supported` + - `NotReadableError`, `NotSupportedError: ... public key credentials` + - `Could not establish connection. Receiving end does not exist.` + - `A listener indicated an asynchronous response by returning true` + - `Invalid call to runtime.sendMessage()` (extension messaging) + - `IndexedDB ... Internal error opening backing store` + - `An error occured reading the Blob argument to createImageBitmap` +- **Examples:** + - `NotAllowedError: Failed to execute 'writeText' on 'Clipboard': Document is not focused.` + - `OperationError: A request is already pending.` + - `SecurityError: Blocked a frame with origin "" from accessing a cross-origin frame.` +- **Reason:** `Browser API permission noise — ; not actionable from our code.` + +## 6. Transient backend 5xx + +Frontend errors that are downstream of intermittent backend failures already triaged elsewhere. + +- **Signals:** title is `InternalServerError: /api/0/...`, `ServiceUnavailableError: /...`, or `UndefinedResponseBodyError: /...`. +- Culprit is in app code that calls our API, but the root cause is a backend 5xx. +- **Examples:** + - `InternalServerError: GET /organizations/{orgSlug}/issues/{issueId}/ 500` + - `ServiceUnavailableError: GET // 503` + - `UndefinedResponseBodyError: GET /organizations/{orgSlug}// 200` +- **Reason:** `Transient backend 5xx — ; backend transient, not a frontend bug.` +- **Caution:** if a single 5xx is firing thousands of times per day to one specific endpoint, surface as `needs-human` — it may signal a real backend regression. + +## 7. Network noise / Failed to fetch + +Network failures to third-party hosts (analytics, CDNs, customer-side services) that we cannot fix from the frontend. + +- **Signals:** title is `TypeError: Failed to fetch ()` where `` is *not* a Sentry-owned domain. +- Common patterns: well-known analytics endpoints, ad/marketing endpoints, customer-internal hostnames (often subdomains with random tokens), and one-off external CDNs. +- **Examples:** + - `TypeError: Failed to fetch (api2.amplitude.com)` (well-known analytics) + - `TypeError: Failed to fetch ()` +- **Reason:** `Network noise — Failed to fetch from ; third-party network.` +- **Caution:** `Failed to fetch (sentry.io)` from within a Sentry context could indicate a real outage. Check recency; if multiple users in the last hour, surface as `needs-human`. + +## 8. Customer-environment proxy interference + +Corporate proxies or content filters that intercept responses, returning HTML notification pages where JSON was expected. + +- **Signals:** title is `Error: 200 treated as error: ...` or `JSON parse error` with response body containing `` / `` / proxy notice text (common products: McAfee, Forcepoint, Zscaler, Symantec). +- **Examples:** + - `Error: 200 treated as error: GET /organizations/{orgSlug}/projects/` +- **Reason:** `Customer-environment proxy interference — corporate proxy returned HTML for JSON request.` + +## 9. Zero-impact / unknown title + +Issues with zero users impacted, very low event count, and no useful title (``, empty, or unparseable). + +- **Signals:** `users == 0` AND `events ≤ 50` AND title is `` or empty. +- **Reason:** `Zero-impact — no users affected, low volume.` +- **Caution:** unknown-title issues at high volume (events ≥ 1000) often hide real bugs that need a frame. Surface as `needs-human` rather than archive. + +--- + +## Negative criteria — do NOT archive + +Even if an issue is high-volume or annoying, **skip** when any of these apply: + +- Top in-app frame is in our application code (`app/`, `static/`, `src/`) — even if the error message is generic. +- Title contains `ZodError` with a stack pointing into our schema validation — likely a real schema mismatch we should fix. +- Title is a React internal error pointing at our component tree (`useEffectEvent`, `hydration mismatch`, render-time exceptions in our components). +- The issue was filed via "User Feedback" — someone took the time to report it. +- Volume jumped recently (e.g., 0 events 7 days ago → 1000 today) — looks like a regression. +- The error is `Error: Should not already be working.` from a React internal *and* it's recurring across many users in our org — escalate. + +When negative criteria fire, mark `skip` with a short note, e.g. `Skip — top frame is our app/, looks actionable.` + +--- + +## High-confidence vs. needs-human decision matrix + +| Top frame location | Title category match | Volume | Decision | +|--------------------|----------------------|--------|----------| +| Third-party (`node_modules//...`) | yes | any | `archive` | +| Third-party | no clean category | any | `needs-human` | +| Our app code | no | any | `skip` | +| `` | n/a | low (≤ 50 events) | `archive` (zero-impact) | +| `` | n/a | high (≥ 1000 events) | `needs-human` | +| Customer proxy / synthetic | yes | low | `archive` | +| Backend 5xx pattern | yes | low/medium | `archive` | +| Backend 5xx pattern | yes | very high single-endpoint | `needs-human` |