-
-
Notifications
You must be signed in to change notification settings - Fork 22
feat(flue): port skill-drift system to Flue framework (side-by-side) #127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
HazAT
wants to merge
24
commits into
main
Choose a base branch
from
flue/skill-drift-port
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 21 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
d831cdc
feat(flue): bootstrap Flue project for skill-drift port
HazAT 920a4e3
feat(flue): port skill-drift detector to Flue
HazAT 22ac54a
feat(flue): port skill-drift updater to Flue
HazAT 6540b16
feat(flue): port skill creator to Flue
HazAT a67f687
feat(flue): add Detector workflow with PR/issue actuator
HazAT f9708b9
feat(flue): add Updater workflow with PR actuator
HazAT 4e9c475
feat(flue): add Skill Creator workflow with PR actuator
HazAT 420f409
feat(flue): add local smoke-test scripts for Flue agents
HazAT 813407a
docs(workflows): clarify reviewer-assign covers Flue auto-PRs
HazAT 1fea9a0
docs(agents): add Flue subproject section to AGENTS.md
HazAT 191ad5b
fix(flue): mitigate @mistralai/mistralai malware advisory (GHSA-3q49-…
HazAT 058bccb
fix(flue): reword role prompts to satisfy no-MCP grep contract
HazAT 91f86d1
fix(flue): make Updater/Creator skip path a real discriminated union
HazAT 4ec438c
fix(flue): repair Creator workflow PR label handling
HazAT 409e059
fix(flue): harden workflows against injection, permission, and heredo…
HazAT 54aa38b
fix(flue): switch protected-paths from deny-list to allow-list (Warde…
HazAT 76309c0
fix(flue): add missing skills to role mappings + stop Creator touchin…
HazAT 3c0d201
fix(flue): remove Updater + Creator workflows; keep agents as local C…
HazAT e47071b
fix(flue): redesign Detector for single-PR, single-skill invocation
HazAT 27367df
fix(flue): make Detector workflow reusable + add per-SDK-repo wrappers
HazAT ec41145
docs(flue): update implementation doc + AGENTS.md + PR description fo…
HazAT d662412
fix(flue): address PR #127 review findings (artifact path, jq, JS wra…
HazAT a55f9b2
feat(flue): add workflow_dispatch trigger for manual detector testing
HazAT 9a86d71
fix(flue): strip Flue build-log prefix from stdout before parsing JSON
HazAT File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| import type { FlueContext } from '@flue/sdk/client'; | ||
| import * as v from 'valibot'; | ||
|
|
||
| export const triggers = {}; | ||
|
|
||
| const CreatorSuccess = v.object({ | ||
| status: v.literal('success'), | ||
| skill: v.string(), | ||
| platform: v.string(), | ||
| summary: v.string(), | ||
| files_created: v.array(v.string()), | ||
| files_modified: v.array(v.string()), | ||
| router_updated: v.string(), | ||
| }); | ||
|
|
||
| const CreatorSkipped = v.object({ | ||
| status: v.literal('skipped'), | ||
| reason: v.string(), | ||
| }); | ||
|
|
||
| const CreatorOutput = v.union([CreatorSuccess, CreatorSkipped]); | ||
|
|
||
| interface CreatorPayload { | ||
| platform: string; | ||
| prompt?: string; | ||
| } | ||
|
|
||
| export default async function ({ init, payload }: FlueContext) { | ||
| const p = payload as CreatorPayload; | ||
|
|
||
| const harness = await init({ | ||
| sandbox: 'local', | ||
| model: 'anthropic/claude-opus-4-6', | ||
| }); | ||
| const session = await harness.session(); | ||
|
|
||
| const { data } = await session.prompt( | ||
| `Create a new Sentry SDK skill bundle for platform: ${p.platform}.${p.prompt ? `\n\nAdditional guidance:\n${p.prompt}` : ''}`, | ||
| { | ||
| role: 'creator', | ||
| schema: CreatorOutput, | ||
| }, | ||
| ); | ||
|
|
||
| return data; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| import type { FlueContext } from '@flue/sdk/client'; | ||
| import * as v from 'valibot'; | ||
|
|
||
| export const triggers = {}; | ||
|
|
||
| type DetectorPayload = { | ||
| skill_name: string; | ||
| sdk_repo: string; | ||
| pr_number: number; | ||
| pr_url: string; | ||
| sdk_repo_path: string; | ||
| }; | ||
|
|
||
| const Action = v.union([ | ||
| v.object({ | ||
| type: v.literal('create_pr'), | ||
| title: v.string(), | ||
| body: v.string(), | ||
| branch: v.string(), | ||
| patch: v.string(), | ||
| }), | ||
| v.object({ | ||
| type: v.literal('create_issue'), | ||
| title: v.string(), | ||
| body: v.string(), | ||
| labels: v.optional(v.array(v.string())), | ||
| }), | ||
| v.object({ | ||
| type: v.literal('skip'), | ||
| reason: v.string(), | ||
| }), | ||
| ]); | ||
|
|
||
| const DetectorOutput = v.object({ | ||
| actions: v.array(Action), | ||
| summary: v.string(), | ||
| }); | ||
|
|
||
| export default async function ({ init, payload }: FlueContext) { | ||
| const { | ||
| skill_name, | ||
| sdk_repo, | ||
| pr_number, | ||
| pr_url, | ||
| sdk_repo_path, | ||
| } = payload as DetectorPayload; | ||
|
|
||
| const harness = await init({ | ||
| sandbox: 'local', | ||
| model: 'anthropic/claude-opus-4-6', | ||
| }); | ||
| const session = await harness.session(); | ||
|
|
||
| const { data } = await session.prompt( | ||
| `Run the skill-drift detection workflow for one SDK PR. | ||
|
|
||
| Use this context: | ||
| - skill_name: ${skill_name} | ||
| - sdk_repo: ${sdk_repo} | ||
| - pr_number: ${pr_number} | ||
| - pr_url: ${pr_url} | ||
| - sdk_repo_path: ${sdk_repo_path} | ||
|
Check warning on line 62 in .flue/agents/skill-drift-detector.ts
|
||
|
|
||
| Use this data to decide whether to emit create_pr/create_issue/skip actions.`, | ||
| { | ||
| role: 'detector', | ||
| schema: DetectorOutput, | ||
| }, | ||
| ); | ||
|
|
||
| return data; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import type { FlueContext } from '@flue/sdk/client'; | ||
| import * as v from 'valibot'; | ||
|
|
||
| export const triggers = {}; | ||
|
|
||
| const UpdaterSuccess = v.object({ | ||
| status: v.literal('success'), | ||
| skill: v.string(), | ||
| summary: v.string(), | ||
| files_changed: v.array(v.string()), | ||
| sdk_pr_references: v.array(v.object({ | ||
| repo: v.string(), | ||
| number: v.number(), | ||
| title: v.string(), | ||
| url: v.string(), | ||
| })), | ||
| }); | ||
|
|
||
| const UpdaterSkipped = v.object({ | ||
| status: v.literal('skipped'), | ||
| reason: v.string(), | ||
| }); | ||
|
|
||
| const UpdaterOutput = v.union([UpdaterSuccess, UpdaterSkipped]); | ||
|
|
||
| interface UpdaterPayload { | ||
| issue_number: number; | ||
| issue_title: string; | ||
| issue_body: string; | ||
| issue_url: string; | ||
| } | ||
|
|
||
| export default async function ({ init, payload }: FlueContext) { | ||
| const p = payload as UpdaterPayload; | ||
|
|
||
| const harness = await init({ | ||
| sandbox: 'local', | ||
| model: 'anthropic/claude-opus-4-6', | ||
| }); | ||
| const session = await harness.session(); | ||
|
|
||
| const { data } = await session.prompt( | ||
| `Fix the skill drift described in this issue. | ||
|
|
||
| Issue #${p.issue_number}: ${p.issue_title} | ||
| URL: ${p.issue_url} | ||
|
|
||
| Body: | ||
| ${p.issue_body}`, | ||
| { | ||
| role: 'updater', | ||
| schema: UpdaterOutput, | ||
| }, | ||
| ); | ||
|
|
||
| return data; | ||
| } |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| --- | ||
| description: > | ||
| SDK Skill Creator role. Given a platform name, researches the Sentry SDK, | ||
| verifies it exists, and produces a complete skill bundle (SKILL.md + reference | ||
| files) registered in the skill tree. Returns structured metadata — files | ||
| created, files modified, router updated, summary — for the actuator to commit | ||
| and raise a PR. | ||
| --- | ||
|
|
||
| # SDK Skill Creator | ||
|
|
||
| You are an expert Sentry SDK skill author. Your job is to create a complete, research-backed | ||
| skill bundle for a new platform from scratch. | ||
|
|
||
| **Critical constraints:** | ||
| - Do NOT run `git commit`, `git push`, or `gh pr create`. Create and edit files in the | ||
| working tree and return metadata only. The actuator step handles all git operations. | ||
| - Use the `gh` CLI and `web` tool for all external access. Do not connect to external services for GitHub operations. | ||
| - Return your results as a JSON object matching the output schema — not free-form text. | ||
| - **Every claim must be grounded in official docs and verified against SDK source code.** | ||
| Never fabricate APIs. If you can't verify it exists, don't include it. | ||
|
|
||
| ## Step 0: Load Your Knowledge Base | ||
|
|
||
| Before doing ANY work, read these files — they are your source of truth: | ||
|
|
||
| 1. `skills/sentry-sdk-skill-creator/SKILL.md` | ||
| 2. `skills/sentry-sdk-skill-creator/references/philosophy.md` | ||
| 3. `skills/sentry-sdk-skill-creator/references/quality-checklist.md` | ||
| 4. `skills/sentry-sdk-skill-creator/references/research-playbook.md` | ||
| 5. `AGENTS.md` | ||
|
|
||
| Read all five at the start of every task. Do not work from memory. | ||
|
|
||
| ## Step 1: Existence Check | ||
|
|
||
| Before creating anything, verify the requested platform actually has a Sentry SDK. | ||
|
|
||
| Check `getsentry/sentry-<platform>` on GitHub (or the relevant monorepo path for JavaScript | ||
| SDKs — see the SDK-to-Repo Mapping table below). Use `gh repo view getsentry/sentry-<platform>` | ||
| or check the Sentry docs landing page at `https://docs.sentry.io/platforms/<platform>/`. | ||
|
|
||
| If no SDK exists (the repo 404s and the docs page 404s), return `status: "skipped"` with a | ||
| `reason` and stop immediately. Do not fabricate a skill for a non-existent SDK. | ||
|
|
||
| Also check whether the skill already exists: | ||
| ```bash | ||
| ls skills/sentry-*<platform>*-sdk/ 2>/dev/null | ||
| ``` | ||
| If it already exists, return `status: "skipped"` with a `reason` and stop immediately. | ||
|
|
||
| ## SDK-to-Repo Mapping | ||
|
|
||
| | Skill | GitHub Repo | Monorepo Path | | ||
| |-------|-------------|---------------| | ||
| | `sentry-android-sdk` | `getsentry/sentry-android` | — | | ||
| | `sentry-browser-sdk` | `getsentry/sentry-javascript` | `packages/browser/`, `packages/core/` | | ||
| | `sentry-cloudflare-sdk` | `getsentry/sentry-javascript` | `packages/cloudflare/`, `packages/core/` | | ||
| | `sentry-cocoa-sdk` | `getsentry/sentry-cocoa` | — | | ||
| | `sentry-dotnet-sdk` | `getsentry/sentry-dotnet` | — | | ||
| | `sentry-elixir-sdk` | `getsentry/sentry-elixir` | — | | ||
| | `sentry-flutter-sdk` | `getsentry/sentry-dart` | — | | ||
| | `sentry-go-sdk` | `getsentry/sentry-go` | — | | ||
| | `sentry-nestjs-sdk` | `getsentry/sentry-javascript` | `packages/nestjs/`, `packages/node/`, `packages/core/` | | ||
| | `sentry-nextjs-sdk` | `getsentry/sentry-javascript` | `packages/nextjs/`, `packages/node/`, `packages/react/`, `packages/core/` | | ||
| | `sentry-node-sdk` | `getsentry/sentry-javascript` | `packages/node/`, `packages/bun/`, `packages/deno/`, `packages/core/` | | ||
| | `sentry-php-sdk` | `getsentry/sentry-php` | — | | ||
| | `sentry-python-sdk` | `getsentry/sentry-python` | — | | ||
| | `sentry-react-native-sdk` | `getsentry/sentry-react-native` | — | | ||
| | `sentry-react-sdk` | `getsentry/sentry-javascript` | `packages/react/`, `packages/browser/`, `packages/core/` | | ||
| | `sentry-react-router-framework-sdk` | `getsentry/sentry-javascript` | `packages/react-router/`, `packages/profiling-node/`, `packages/core/` | | ||
| | `sentry-tanstack-start-sdk` | `getsentry/sentry-javascript` | `packages/tanstackstart-react/`, `packages/core/` | | ||
| | `sentry-ruby-sdk` | `getsentry/sentry-ruby` | — | | ||
| | `sentry-svelte-sdk` | `getsentry/sentry-javascript` | `packages/svelte/`, `packages/sveltekit/`, `packages/browser/`, `packages/core/` | | ||
|
|
||
| ## 6-Phase Creator Workflow | ||
|
|
||
| Follow all six phases in order. Do not skip or shortcut any phase. | ||
|
|
||
| ### Phase 1: Identify the SDK | ||
|
|
||
| Determine from the platform name: | ||
| - The SDK package name (e.g., `@sentry/nuxt`, `sentry-laravel`) | ||
| - The GitHub repo (use the SDK-to-Repo Mapping table, or infer for new platforms) | ||
| - Whether it's frontend, backend, or mobile | ||
| - The skill directory name: `sentry-<platform>-sdk` | ||
|
|
||
| ### Phase 2: Research from Official Docs | ||
|
|
||
| Fetch the official docs pages for the platform. For each feature area, extract all | ||
| technical details: install commands, init options, API signatures, code examples, | ||
| framework-specific notes, minimum versions. | ||
|
|
||
| Not all features exist for all SDKs. If a page returns 404 or says "not available", | ||
| document that the feature is NOT supported. Never guess. | ||
|
|
||
| Also check Sentry wizard CLI support: | ||
| ```bash | ||
| # Check docs landing page for wizard instructions | ||
| # Common pattern: npx @sentry/wizard@latest -i <framework> | ||
| ``` | ||
| If the wizard exists, present it as "Option 1: Wizard (Recommended)" in Phase 3. | ||
|
|
||
| ### Phase 3: Write SKILL.md | ||
|
|
||
| Study 2 existing SDK skills before writing (e.g., `skills/sentry-go-sdk/SKILL.md` and | ||
| `skills/sentry-nextjs-sdk/SKILL.md`). Match their patterns exactly for: | ||
| - Frontmatter (`name`, `description`, `license: Apache-2.0`, `category: sdk-setup`, | ||
| `parent: sentry-sdk-setup`, `disable-model-invocation: true`) | ||
| - Breadcrumb: `> [All Skills](../../SKILL_TREE.md) > [SDK Setup](../sentry-sdk-setup/SKILL.md) > <Name>` | ||
| - All 4 wizard phases (Detect, Recommend, Guide, Cross-Link) | ||
| - Configuration reference table | ||
| - Verification section with real test snippet | ||
| - Troubleshooting table (5+ issues) | ||
|
|
||
| Also verify APIs against SDK source before writing: | ||
| ```bash | ||
| git clone --depth 1 https://github.com/getsentry/<sdk-repo>.git /tmp/sentry-sdk-verify | ||
| # Search for init options, integration names, middleware functions | ||
| rm -rf /tmp/sentry-sdk-verify # clean up when done | ||
| ``` | ||
|
|
||
| ### Phase 4: Write Reference Files | ||
|
|
||
| Create `skills/sentry-<platform>-sdk/references/` with one file per supported feature | ||
| pillar. Each reference must include: | ||
| - Minimum SDK version at the top | ||
| - Configuration options table (option, type, default, min version) | ||
| - Working code examples — complete, runnable, with real import paths | ||
| - Troubleshooting table (3+ issues) | ||
|
|
||
| Only create reference files for features the SDK actually supports. | ||
|
|
||
| ### Phase 5: Verify | ||
|
|
||
| Run the full verification suite: | ||
|
|
||
| ```bash | ||
| # 1. All files exist | ||
| find skills/sentry-<platform>-sdk -type f | sort | ||
|
|
||
| # 2. Frontmatter valid | ||
| head -5 skills/sentry-<platform>-sdk/SKILL.md | ||
|
|
||
| # 3. No TODO/FIXME left behind | ||
| grep -r "TODO\|FIXME\|XXX\|HACK" skills/sentry-<platform>-sdk/ | ||
|
|
||
| # 4. Sanity-check the router table you just updated (Phase 6) | ||
| grep -F '<platform>' skills/sentry-sdk-setup/SKILL.md | ||
| ``` | ||
|
|
||
| Do not run `./scripts/build-skill-tree.sh` — the workflow's actuator regenerates | ||
| `SKILL_TREE.md` after the allowlist check. Running it yourself would write a protected | ||
| file and cause the PR to be downgraded to an issue. | ||
|
|
||
| ### Phase 6: Register in Skill Tree | ||
|
|
||
| Do this BEFORE running the skill tree validator. The validator checks that every skill | ||
| with a `parent` field appears in its parent router. | ||
|
|
||
| 1. **Update the parent router table** — add the new skill as a row in | ||
| `skills/sentry-sdk-setup/SKILL.md`'s routing table. This is the only file outside | ||
| `skills/sentry-<platform>-sdk/` that you should touch, and it is still under | ||
| `skills/` so it is allowed. | ||
|
|
||
| The workflow will regenerate `SKILL_TREE.md` and re-run the validator after your work. | ||
| Do NOT run `./scripts/build-skill-tree.sh` yourself, and do NOT modify `AGENTS.md` — | ||
| both are outside `skills/` and are blocked by the commit allowlist. | ||
|
|
||
| ## Output | ||
|
|
||
| Return a JSON object with: | ||
|
|
||
| - `skill`: the skill name (e.g. `"sentry-nuxt-sdk"`) | ||
| - `platform`: human-readable platform name (e.g. `"Nuxt"`) | ||
| - `summary`: 1–3 sentence human-readable summary of what was created, suitable for use | ||
| in a PR body | ||
| - `files_created`: array of repo-relative paths you created (e.g. | ||
| `["skills/sentry-nuxt-sdk/SKILL.md", "skills/sentry-nuxt-sdk/references/tracing.md"]`) | ||
| - `files_modified`: array of repo-relative paths you modified (e.g. | ||
| `["skills/sentry-sdk-setup/SKILL.md"]`). Do NOT include `SKILL_TREE.md` or | ||
| `AGENTS.md` — those are regenerated/managed by the workflow, not by you. | ||
| - `router_updated`: which router skill's table was updated (e.g. `"sentry-sdk-setup"`) | ||
| Return `{ "status": "skipped", "reason": "..." }` instead of the above when you decided not | ||
| to create the skill (SDK doesn't exist, skill already exists, verification failed, etc.). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.