-
Notifications
You must be signed in to change notification settings - Fork 39
feat(triage-frontend-issues): Add archive-only triage skill for sentry/javascript #151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| --- | ||
| name: triage-frontend-issues | ||
| description: Triage new issues in the Sentry `javascript` project by archiving non-actionable noise. Use when asked to "triage issues", "triage the javascript project", "archive non-actionable issues", "triage new frontend issues", or "clean up the sentry/javascript queue". Operates only on the sentry/javascript project, only archives (never resolves), and always archives with `untilEscalating`. | ||
| allowed-tools: Read, mcp__sentry__search_issues, mcp__sentry__get_sentry_resource, mcp__sentry__update_issue | ||
| --- | ||
|
|
||
| # Triage Frontend Issues | ||
|
|
||
| Archive non-actionable noise from the `sentry/javascript` issue queue: only archive, always `untilEscalating`, always with a stated reason. Issues that look actionable in our code, or that you cannot confidently classify, must be skipped. | ||
|
|
||
| ## Hard Rules | ||
|
|
||
| These rules override anything else. Do not relax them. | ||
|
|
||
| 1. **Project scope.** Only operate on `organizationSlug=sentry`, project slug `javascript`. If asked to triage a different project, stop and ask the user to confirm. | ||
| 2. **Archive only.** The only status mutation permitted is `status=ignored`. Never resolve, never unresolve, never assign, never delete, never bulk-update fields other than status. | ||
| 3. **Always `untilEscalating`.** Use `ignoreMode=untilEscalating`. Never use `forever`, `forDuration`, `untilOccurrenceCount`, or `untilUserCount`. If the user asks for a different mode, stop and have them archive that issue manually — this skill does not perform non-escalating archives. | ||
| 4. **Always include a `reason`.** The `reason` must be a short, factual sentence naming the category from `references/archive-criteria.md` (e.g., "Third-party library noise — echarts internals; not actionable in our code"). | ||
| 5. **Never touch issues outside the unresolved queue.** Skip anything with `status` of `resolved`, `ignored`, or `reprocessing`. | ||
| 6. **Never archive without confirmation.** Build a full plan, show it to the user, wait for explicit approval before calling `update_issue`. A single approval covers the displayed plan only; new batches need new approval. | ||
| 7. **When in doubt, skip.** If an issue could plausibly be a real bug in our code, do not archive it. Surface it as `needs-human` in the plan with a one-line note. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Sentry MCP authenticated via `mcp.sentry.dev`. Required tools: `search_issues`, `get_sentry_resource`, `update_issue`. | ||
| - If `update_issue` is not available, stop and ask the user to authenticate the Sentry MCP server. | ||
|
|
||
| ## Inputs | ||
|
|
||
| `$ARGUMENTS` is one of: | ||
|
|
||
| | Input shape | Meaning | | ||
| |-------------|---------| | ||
| | Sentry issue URL (`https://sentry.sentry.io/issues/JAVASCRIPT-…`) | Triage that single issue. | | ||
| | Issue short ID (`JAVASCRIPT-…`) | Triage that single issue. | | ||
| | Sentry issue query (contains a colon, e.g. `is:unresolved firstSeen:-24h`) | Use as the search query. | | ||
| | Empty | Use the default triage queue: `is:unresolved is:unassigned firstSeen:-7d`, sort `new`, limit `50`. | | ||
|
|
||
| If `$ARGUMENTS` is ambiguous, ask the user to clarify before searching. | ||
|
|
||
| ## Workflow | ||
|
|
||
| ### 1. Load the queue | ||
|
|
||
| For single-issue input: | ||
| - Call `get_sentry_resource(url=<issue-url>)` or `get_sentry_resource(resourceType='issue', organizationSlug='sentry', resourceId=<shortId>)`. | ||
| - Confirm `Project` is the javascript frontend project. If not, stop. | ||
|
|
||
| For query/default input: | ||
| - Call `search_issues(organizationSlug='sentry', projectSlugOrId='javascript', query=<query>, sort='new', limit=50)`. | ||
| - Then call `get_sentry_resource` for each result in parallel to get culprit, substatus, assignee, and stack-frame hints (the search response omits some fields). | ||
|
|
||
| Skip immediately if any of these are true on an issue: | ||
|
|
||
| - `status` is not `unresolved` (already archived, resolved, or in reprocessing). | ||
| - `assignedTo` is set to a human (someone is already owning it). | ||
| - `assignedTo` is set to a team other than `frontend`/`issues` and the issue looks team-specific (let the owning team triage). | ||
|
|
||
| ### 2. Classify each issue | ||
|
|
||
| Read `references/archive-criteria.md` for the category taxonomy with recognition heuristics and examples. For each candidate issue, produce one of: | ||
|
|
||
| | Decision | Meaning | | ||
| |----------|---------| | ||
| | `archive` | Matches a documented category; include the category name in the reason. | | ||
| | `skip` | Could be a real bug in our code, or insufficient evidence; do not archive. | | ||
| | `needs-human` | Looks like noise but doesn't cleanly fit a category, or volume is unusually high; flag for user review. | | ||
|
|
||
| When evaluating, weight these signals (in this order): | ||
|
|
||
| 1. **Top non-Sentry-SDK frame.** If the top in-app frame is in `node_modules/`, `chrome-extension://`, a third-party host, or `<unknown>`, this is a strong archive signal. | ||
| 2. **Title pattern.** Many archives are recognizable from the title alone (see criteria reference). | ||
| 3. **Volume is not a veto.** Some high-volume issues (10k+ events, thousands of users) are still archive-worthy if the top frame is third-party. Volume alone never forces archive either. | ||
| 4. **Recency.** Single-event issues older than 30 days with no recurrence are usually noise. | ||
| 5. **Customer org spread.** If events come from one customer subdomain only (check `customerDomain.subdomain` tag), it is likely customer-environment noise. | ||
|
|
||
| ### 3. Build the plan | ||
|
|
||
| Output one Markdown table to the user, in this exact shape: | ||
|
|
||
| ``` | ||
| ## Triage plan — sentry/javascript (<N> candidates) | ||
|
|
||
| | # | Issue | Title | Volume | Decision | Category | Reason | | ||
| |---|-------|-------|--------|----------|----------|--------| | ||
| | 1 | [JAVASCRIPT-XXXX](url) | TypeError: ... | 12e/3u | archive | browser-api-noise | Browser clipboard permission denied; not actionable. | | ||
| | 2 | [JAVASCRIPT-YYYY](url) | <unknown> | 4945e/123u | needs-human | — | High volume, no title — please review before archiving. | | ||
| | 3 | [JAVASCRIPT-ZZZZ](url) | ZodError: ... | 360e/132u | skip | — | Schema validation failure in our code; looks actionable. | | ||
| ``` | ||
|
|
||
| Then summarize counts: `N archive / M skip / K needs-human`. End with: | ||
|
|
||
| ``` | ||
| Reply `apply` to archive the N issues marked `archive`, `apply N,M,...` to archive a subset, or `cancel` to take no action. | ||
| ``` | ||
|
|
||
| ### 4. Apply on approval | ||
|
|
||
| When the user replies `apply` (or `apply <subset>`): | ||
|
|
||
| For each issue in the approved set, call: | ||
|
|
||
| ``` | ||
| update_issue( | ||
| organizationSlug='sentry', | ||
| issueId=<shortId>, | ||
| status='ignored', | ||
| ignoreMode='untilEscalating', | ||
| reason=<category-tagged reason from the plan>, | ||
| ) | ||
| ``` | ||
|
|
||
| Run these sequentially (not in parallel). If a call fails, log the failure, continue with the remaining issues, and report the failed IDs in step 5. | ||
|
|
||
| If the user replies `cancel` or asks to modify the plan, do NOT call `update_issue`. If they reply with edits ("change row 2 to skip"), rebuild the plan and re-confirm. | ||
|
|
||
| ### 5. Report | ||
|
|
||
| After applying, output: | ||
|
|
||
| ``` | ||
| ## Triage report | ||
|
|
||
| - Archived: N | ||
| - Skipped: M | ||
| - Needs human review: K | ||
| - Failures: F (with issue IDs) | ||
|
|
||
| <details><summary>Archived issues</summary> | ||
|
|
||
| - JAVASCRIPT-XXXX — <reason> | ||
| - ... | ||
|
|
||
| </details> | ||
| ``` | ||
|
|
||
| ## Recovery | ||
|
|
||
| - If `update_issue` fails on one item, log the failure and continue with the rest. Report failed IDs at the end. | ||
| - If the user notices a wrong archive, the user can unarchive it themselves in Sentry. The skill never reverses its own actions automatically. | ||
| - If the user asks "redo the plan with these tweaks" mid-flow, regenerate the plan from scratch — do not assume the previous plan still applies. | ||
|
|
||
| ## Example reasons (use this voice) | ||
|
|
||
| - `Third-party library noise — echarts tooltip; not actionable in our code.` | ||
| - `Browser API permission noise — Clipboard writeText denied by user agent.` | ||
| - `Customer-environment proxy interference — 200 response treated as error (HTML body from corporate proxy).` | ||
| - `Transient backend 5xx — InternalServerError on /api/0/organizations/.../events-meta/; backend transient.` | ||
| - `Test/synthetic event — smoke test or security probe, not production traffic.` | ||
| - `Wrong project — Prisma/Python error mis-routed to frontend project.` | ||
| - `Single-event fluke — 1 event, 1 user, no recurrence in 30+ days.` | ||
| - `Browser extension noise — ReferenceError for extension-injected global (DarkReader/WeixinJSBridge).` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # SPEC — triage-frontend-issues | ||
|
|
||
| ## Intent | ||
|
|
||
| Reduce noise in the `sentry/javascript` issue queue by archiving issues whose root cause is outside our code: third-party libraries, browser/OS quirks, customer-environment interference, transient backend 5xx, test/synthetic events, and single-event flukes. | ||
|
|
||
| The skill never resolves, never assigns, never deletes — only archives, only with `untilEscalating`, only after explicit user approval. | ||
|
|
||
| ## Scope | ||
|
|
||
| - **In scope:** the Sentry `javascript` project under the `sentry` organization. Archive-only triage of unresolved issues. | ||
| - **Out of scope:** any other project. Any non-archive status change (resolve, assign, delete, escalate). Any auto-apply mode without confirmation. Any change to issue ownership routing. Anything outside Sentry MCP. | ||
|
|
||
| ## Trigger Context | ||
|
|
||
| - User explicitly invokes the skill (`/triage-frontend-issues`) or natural language: "triage the javascript project", "archive non-actionable issues", "clean up the unresolved queue". | ||
| - User pastes a `sentry/javascript` issue URL or short ID and asks for a triage decision. | ||
|
|
||
| ## Source / Evidence Model | ||
|
|
||
| The category taxonomy in `references/archive-criteria.md` was synthesized from observed historical archive activity in the `sentry/javascript` project. The activity feed for archived issues was fetched and inspected to determine: | ||
|
|
||
| - which `ignoreMode` was used (consistent across the population — drove Hard Rule 3 in `SKILL.md`) | ||
| - which kinds of issues are archived versus resolved | ||
| - which top-frame and title patterns reliably correspond to non-actionable noise | ||
|
|
||
| Boundary cases were established by inspecting `set_resolved` actions in the same project to identify what the team treats as actionable. Specific counts and per-engineer activity are not captured in this repository. | ||
|
|
||
| ## Reference Architecture | ||
|
|
||
| ``` | ||
| triage-frontend-issues/ | ||
| ├── SKILL.md # runtime instructions (router) | ||
| ├── SPEC.md # this file (maintenance contract) | ||
| └── references/ | ||
| └── archive-criteria.md # category taxonomy with recognition heuristics | ||
| ``` | ||
|
|
||
| Runtime always reads `SKILL.md`. The criteria reference is loaded during step 2 (classification). SPEC is not loaded at runtime. | ||
|
|
||
| ## Evaluation Expectations | ||
|
|
||
| A future iteration of this skill should be evaluated against: | ||
|
|
||
| 1. **Precision on positive cases.** Re-run the skill against historical archived issues and confirm it would archive them with a matching category. | ||
| 2. **Precision on negative cases.** Re-run against historical resolved issues. The skill must mark these `skip`, never `archive`. | ||
| 3. **Reason quality.** Sampled archive `reason` strings should name a category from the taxonomy and identify a concrete signal (library, API, endpoint). | ||
| 4. **No-op safety.** With `cancel` reply, no `update_issue` calls are made. | ||
|
|
||
| Hold-out evaluation set should be maintained at `references/evidence/` (not created yet; add when the first false positive is observed). | ||
|
|
||
| ## Known Limitations | ||
|
|
||
| - The skill reads issue metadata via Sentry MCP, but the MCP's `get_sentry_resource` output does not surface the activity feed. | ||
| - Determining "who last touched this issue" requires HTTP API calls, which the skill does not perform — it defers to the user when an issue's history matters. | ||
| - The skill cannot inspect the full stack trace in machine-readable form; it relies on what `get_sentry_resource` returns in its formatted output. | ||
| - Edge cases where the top frame is buried may be misclassified — those are intentionally routed to `needs-human`. | ||
| - Title pattern matching is heuristic. Novel third-party libraries or browser quirks will not match existing patterns and should be added to `references/archive-criteria.md` after the first occurrence. | ||
| - The category taxonomy is sourced from a single project (`sentry/javascript`). Applying it to a different project requires re-validating the categories. | ||
|
|
||
| ## Maintenance Notes | ||
|
|
||
| - When the user flags a misclassification, capture the example, decide whether the criteria need updating, and update `references/archive-criteria.md`. | ||
| - Re-validate the criteria periodically by sampling recent archives and confirming the patterns still apply. | ||
| - If Sentry adds new `ignoreMode` options or changes the `update_issue` API surface, update the Hard Rules and the `update_issue` call snippet in `SKILL.md`. | ||
| - Do not expand the skill's scope beyond archiving. Resolution, assignment, and deletion belong to other workflows. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.