Merge pull request #2203 from modelcontextprotocol/v2/docs/2202-skill-eval-tuning

docs: tune skill eval cases and document how to write them

Author: cliffhall

.claude/skills/board-ops/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: board-ops
-description: gh recipes for the Inspector project boards — add a card, move its Status, set or re-score its Priority, delete a card, and recover from a deleted single-select option. Covers board #28 (v2) and board #11 (v1), their node/field/option IDs, and the option-deletion hazard.
+description: "gh recipes for the Inspector project boards — add a card, move its Status, set or re-score its Priority, delete a card, and recover from a deleted single-select option. Covers board #28 (v2) and board #11 (v1), their node/field/option IDs, and the option-deletion hazard."
 disable-model-invocation: false
 ---
 

.claude/skills/board-ops/evals/evals.json

@@ -11,6 +11,14 @@
     "prompt": "The project board rejected my Status option ID. How do I get the current ones?",
     "expect": "board-ops"
   },
+  {
+    "prompt": "I'm about to edit the options on one of the board's single-select fields. Should I do anything first?",
+    "expect": "board-ops"
+  },
+  {
+    "prompt": "Several cards lost their Priority value after the field was edited. How do I put it back?",
+    "expect": "board-ops"
+  },
   {
     "prompt": "Explain the difference between TCP and UDP.",
     "expect": null

.claude/skills/issue-create/evals/evals.json

@@ -11,6 +11,14 @@
     "prompt": "Before I file this, how do I check nobody has reported it already?",
     "expect": "issue-create"
   },
+  {
+    "prompt": "I found a bug while working on something else. How do I get it tracked in this repo?",
+    "expect": "issue-create"
+  },
+  {
+    "prompt": "What do I need to do beyond `gh issue create` for a new issue to be properly tracked here?",
+    "expect": "issue-create"
+  },
   {
     "prompt": "What is the chemical symbol for potassium?",
     "expect": null

.claude/skills/issue-triage/evals/evals.json

@@ -4,13 +4,21 @@
     "expect": "issue-triage"
   },
   {
-    "prompt": "What Priority should issue 2189 be given?",
+    "prompt": "How do I decide what Priority an incoming issue deserves?",
     "expect": "issue-triage"
   },
   {
     "prompt": "Audit the project board for anything that has drifted.",
     "expect": "issue-triage"
   },
+  {
+    "prompt": "How do I find the issues that aren't on either board yet?",
+    "expect": "issue-triage"
+  },
+  {
+    "prompt": "After scoring an issue against the rubric, what do I do with the score?",
+    "expect": "issue-triage"
+  },
   {
     "prompt": "Who wrote the novel Moby-Dick?",
     "expect": null

.claude/skills/local-dev/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: local-dev
-description: Getting the Inspector running locally, and the reasoning behind its dependency rules. Which install to run and which script starts each client (web, cli, tui, launcher); why v2 is not an npm workspace; how the @inspector/core alias works; and why the placement rules in AGENTS.md are what they are — what each one is defending against, what it looked like when it was violated, and how to tell you have hit one.
+description: Install, build and run the Inspector locally, and place a dependency correctly. Use when starting the web, cli, tui or launcher client; when asking whether a build or rebuild is needed before a run script works; when an install, an @inspector/core import or a client bundle fails at run time; when adding or bumping a dependency and choosing which package.json declares it; or when a bundler external, a version skew or a git worktree is the suspect.
 disable-model-invocation: false
 ---
 

.claude/skills/local-dev/evals/evals.json

@@ -1,10 +1,10 @@
 [
   {
-    "prompt": "How do I get the web client running locally?",
+    "prompt": "What is the difference between the launcher-driven run scripts and starting a client directly, and when should I use each?",
     "expect": "local-dev"
   },
   {
-    "prompt": "I added a dependency and the TUI bundle broke at import time. Where should it have gone?",
+    "prompt": "I added a package and now one of the client bundles fails at run time. What do I check?",
     "expect": "local-dev"
   },
   {
@@ -15,6 +15,10 @@
     "prompt": "How do I run the CLI from the built launcher in this repo instead of via npx?",
     "expect": "local-dev"
   },
+  {
+    "prompt": "Do I need to do anything special to set this repo up inside a git worktree?",
+    "expect": "local-dev"
+  },
   {
     "prompt": "Write a haiku about autumn.",
     "expect": null

.claude/skills/pr-flow/evals/evals.json

@@ -11,6 +11,14 @@
     "prompt": "The DCO check is red on my branch. How do I fix the commits I already pushed?",
     "expect": "pr-flow"
   },
+  {
+    "prompt": "Copilot left a pile of review comments on my PR. How do I answer them so nothing gets lost?",
+    "expect": "pr-flow"
+  },
+  {
+    "prompt": "My PR is open and green. Walk me through getting it merged and closed out here.",
+    "expect": "pr-flow"
+  },
   {
     "prompt": "What does this regex match? /^[a-z]+$/",
     "expect": null

.claude/skills/pre-push-gate/evals/evals.json

@@ -4,13 +4,21 @@
     "expect": "pre-push-gate"
   },
   {
-    "prompt": "The Storybook stage of the gate fails in my worktree even though the tests pass. What is wrong?",
+    "prompt": "The gate's Storybook stage is failing for me. How do I diagnose it?",
     "expect": "pre-push-gate"
   },
   {
     "prompt": "Everything in the gate started timing out at once. What causes that?",
     "expect": "pre-push-gate"
   },
+  {
+    "prompt": "How do I check whether a stage of the gate really passed?",
+    "expect": "pre-push-gate"
+  },
+  {
+    "prompt": "The gate failed at the lint stage. How do I work out what is wrong?",
+    "expect": "pre-push-gate"
+  },
   {
     "prompt": "What is 17 multiplied by 23?",
     "expect": null

.claude/skills/project-structure/evals/evals.json

@@ -1,14 +1,22 @@
 [
   {
-    "prompt": "What lives in clients/web versus @inspector/core?",
+    "prompt": "Something in the web client's Node backend needs changing but I do not know which piece handles what. How do I work that out?",
     "expect": "project-structure"
   },
   {
-    "prompt": "I'm adding a helper both the CLI and the web client need. Which directory does it belong in?",
+    "prompt": "I am adding a module that both the browser side and the Node side need to run. How do I decide where it goes in this repo?",
     "expect": "project-structure"
   },
   {
-    "prompt": "Which package owns the OAuth secret storage backends?",
+    "prompt": "Does browser-only transport code belong in core or in the web client?",
+    "expect": "project-structure"
+  },
+  {
+    "prompt": "I have a new file and I'm not sure where it goes. How do I decide in this repo?",
+    "expect": "project-structure"
+  },
+  {
+    "prompt": "I need to find where a feature is implemented but I do not know which client owns it. How do I work that out in this repo?",
     "expect": "project-structure"
   },
   {

.claude/skills/test-servers/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: test-servers
-description: Run a composable MCP test server by hand — pick the showcase config that reproduces a given feature or bug, build test-servers, and connect with the right protocol era. Covers the stale-build hazard.
+description: Run a composable MCP test server by hand — pick the showcase config for a feature or bug, build it, and connect with the right protocol era. Use when a change, a PR or a smoke test needs a real server to exercise it; when reproducing a reported bug by hand; when choosing which fixture or protocol era to run; when a fixture keeps serving stale code after an edit; or when the config or preset you need does not exist yet.
 disable-model-invocation: false
 ---