feat: skill-creator projection, drift report, in-the-wild inventory, MCP split - #74
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (18)
📝 WalkthroughWalkthroughAdds canonical documentation for three skills and MCP/reporting artifacts, updates the skill-creator workflow, projects skills into core and MCP runtime directories, and records corresponding dispatch-state hashes. ChangesCanonical skill and dispatch updates
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0563ac147a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - Create a few test prompts and run claude-with-access-to-the-skill on them. | ||
| - Help the user evaluate the results both qualitatively and quantitatively. | ||
| - Rewrite the skill based on feedback. | ||
| - Run `git add .` and `git commit` to trigger the dispatch hook, then `git push` to a new branch and open a PR. |
There was a problem hiding this comment.
Do not end the authoring workflow after dispatch and commit: dispatch.sh sync only writes runtime projections and dispatch sentinels; it does not update marketplace.json or capabilities.generated.json. Following this workflow leaves every newly authored skill undiscoverable and untoggleable through /market and without the provenance record required for verify-on-enable—the new retrospect and chittygws skills in this commit already exhibit that state.
Useful? React with 👍 / 👎.
| ```bash | ||
| # Count actual user turns | ||
| grep -c '"type":"USER_INPUT"' $TRANSCRIPT_PATH |
There was a problem hiding this comment.
Resolve or initialize TRANSCRIPT_PATH before executing this validation. Neither supported projection defines this variable, so in a normal Claude Code or Codex invocation the grep reads stdin indefinitely and the following Python opens an empty filename; ARTIFACTS_DIR is likewise undefined, causing the final save path to collapse to the filesystem root. This prevents the skill's central evidence-validation and output steps from completing.
Useful? React with 👍 / 👎.
| canonical/skills/<skill-name>.md (required) | ||
| └── Bundled Resources (optional, placed appropriately) | ||
| ├── scripts/ - Executable code | ||
| ├── references/ - Docs loaded into context | ||
| └── assets/ - Files used in output |
There was a problem hiding this comment.
Specify a real per-skill canonical resource layout and make dispatch copy it. canonical/skills/<skill-name>.md is a file and cannot contain the depicted scripts/, references/, or assets/ children, while the current dispatch adapters project only that Markdown file. Consequently, any skill authored with the optional resources described here loses them from both runtime projections and fails when its instructions reference those files.
Useful? React with 👍 / 👎.
| ### Step 4: Grade, aggregate, and launch the viewer | ||
| 1. **Grade each run** — spawn a grader subagent. | ||
| 2. **Aggregate into benchmark** — run the aggregation script. | ||
| 3. **Do an analyst pass** — read the benchmark data. | ||
| 4. **Launch the viewer** with both qualitative outputs and quantitative data. |
There was a problem hiding this comment.
Provide concrete commands and bundled implementations for aggregation and the viewer. A repository-wide search finds no benchmark aggregation script, viewer, or feedback generator in either skill-creator projection or elsewhere in this repository, so every evaluation reaches this step with nothing it can execute and no component capable of producing the subsequently required feedback.json.
Useful? React with 👍 / 👎.
| ### Cloudflare Access JWT Validation | ||
|
|
||
| When the MCP Portal makes a request to the ChittyGWS MCP endpoints (e.g., `/mcp`), it MUST include a valid Cloudflare Access JWT in the `Cf-Access-Jwt-Assertion` header. |
There was a problem hiding this comment.
Do not instruct the MCP Portal to supply Cf-Access-Jwt-Assertion. The existing auth contract in canonical/agents/chittyagent-connect.md states that identity is injected by Cloudflare Access; the external client authenticates to Access, which then adds the assertion on the origin request. Telling clients or chittyconnect to pass this header both sends operators toward an ineffective fix for 401 responses and risks treating a client-controlled assertion as trusted rather than validating an Access-injected one.
Useful? React with 👍 / 👎.
| | `skill-creator` | 41 (stub) | 480 | `~/.gemini/skills/skill-creator/SKILL.md` | Update canonical to the full projection (completed in this PR), which overrides the 30-line stubs. | | ||
| | `chitty-registry` | 132 | 132 | `~/.gemini/config/skills/chitty-registry/SKILL.md` | The `chittyos-devops` plugin version only has 94 lines. Reconcile latest 132-line version to canonical and dispatch. | | ||
| | `chitty-deploy` | 0 (Wait, is it in canonical?) | 81 | `~/.gemini/config/plugins/chittyos-devops/skills/chitty-deploy/SKILL.md` | Promote 81-line version to canonical, run dispatch to overwrite the 77-line versions. | |
There was a problem hiding this comment.
Regenerate these rows from the committed tree before publishing the report. At this commit canonical/skills/skill-creator.md has 156 lines, not the reported 41-line stub, and canonical/skills/chitty-deploy.md already exists with 85 lines, so the latter recommendation would promote a shorter 81-line wild copy over the richer canonical definition. Operators following this report can therefore perform unnecessary reconciliation or discard canonical content.
Useful? React with 👍 / 👎.
| - Create a few test prompts and run claude-with-access-to-the-skill on them. | ||
| - Help the user evaluate the results both qualitatively and quantitatively. | ||
| - Rewrite the skill based on feedback. | ||
| - Run `git add .` and `git commit` to trigger the dispatch hook, then `git push` to a new branch and open a PR. |
There was a problem hiding this comment.
Replace git add . with an explicit list of canonical, projected, registry, and sentinel files. In the documented flow, evaluation has already created an unignored <skill-name>-workspace/ containing run outputs, timing data, and feedback; staging the whole worktree therefore commits those generated artifacts along with any unrelated user changes or unignored local files.
Useful? React with 👍 / 👎.
| - **Write a draft of the skill into `canonical/skills/<skill-name>.md`**. The ChittyMarket dispatch hook will project this to the appropriate runtime folders. | ||
| - Create a few test prompts and run claude-with-access-to-the-skill on them. | ||
| - Help the user evaluate the results both qualitatively and quantitatively. | ||
| - Rewrite the skill based on feedback. | ||
| - Run `git add .` and `git commit` to trigger the dispatch hook, then `git push` to a new branch and open a PR. |
There was a problem hiding this comment.
Do not rely on git commit to auto-project the canonical. The repository's normal bootstrap does not install the dispatch pre-commit hook—installation requires running plugins/chittyagent-dispatch/scripts/install-hook.sh separately—and canonical/README.md explicitly requires dispatch.sh sync. On a normal checkout this sequence commits no projections; even with the hook installed, the hook creates unstaged files and deliberately blocks the first commit, so the documented one-shot flow still fails.
Useful? React with 👍 / 👎.
| - **Name collision:** Reject if a skill with the same name exists anywhere in canonical/ or if it collides with a known active runtime skill. | ||
| - **Wrong bucket/plugin:** Reject if the skill doesn't fit the designated plugin or if the user doesn't specify a valid plugin bucket. | ||
| - **Already in canonical:** If the skill is already in canonical, you must edit the canonical file, not create a new one. |
There was a problem hiding this comment.
Do not reject an authoring request merely because the user did not supply a plugin bucket. The skill description intentionally triggers on ordinary requests such as “create a skill,” while this binding rule rejects those requests before the following plugin-selection guide can determine the appropriate owner. Infer a bucket from the requested capability or ask for clarification, and reserve refusal for a confirmed invalid selection.
Useful? React with 👍 / 👎.
1 participant
canonical/skills/skill-creator.md— full 480-line ChittyOS-aware projectioncanonical/skills/drift-report.md— top 5 version drift casescanonical/skills/in-the-wild.md— 60+ unregistered skillscanonical/mcp/mcp-split.md— MCP config consolidation planSummary by CodeRabbit
New Features
Documentation