Skip to content

docs(agents): add fleet-wide skill authoring standard - #12

Merged
ruby-dlee merged 5 commits into
mainfrom
fm/skill-standard-q4
Jul 22, 2026
Merged

docs(agents): add fleet-wide skill authoring standard#12
ruby-dlee merged 5 commits into
mainfrom
fm/skill-standard-q4

Conversation

@ruby-dlee

Copy link
Copy Markdown
Owner

Intent

Author firstmate's durable fleet-wide skill-authoring standard so skill crews produce correct first drafts instead of relying on review to catch bloat, duplication, vague triggers, wrong-artifact choices, or brittle specifics. Audit the generic skill creator, firstmate coding guidelines, a project review-time skill audit, and the project's mechanical metadata validator honestly; preserve existing ownership by cross-reference rather than duplication. The standard must gate facts and invariants into AGENTS.md or docs, enforce one skill and one PR per concern, teach principle-level content, make descriptions concrete load triggers, and provide a tight in-draft pre-PR rubric without replacing adversarial review. Keep it self-contained, agent-only, non-user-invocable, at or below 120 lines, add only the required AGENTS.md section 13 load trigger and one firstmate-coding-guidelines cross-pointer, and avoid unrelated edits or project-specific references in this shared template.

What Changed

  • Add an agent-only, fleet-wide skill-authoring standard covering artifact gating, single-concern scope, durable guidance, concrete load triggers, and an in-draft pre-PR rubric.
  • Register the standard in Firstmate’s agent-only skill routing, including project-crew briefing with a path resolved from the active instruction repository.
  • Cross-reference the standard from the existing Firstmate coding guidelines while preserving each skill’s ownership boundaries.

Risk Assessment

✅ Low: The change is narrowly scoped, satisfies the stated authoring-standard requirements, and now routes project crews to the correct absolute skill path from the instruction repository rather than the operational FM_HOME.

Testing

The provided full baseline was green; focused manual review and an end-to-end evidence check verified exact three-file scope, the 73-line cap, internal/non-user-invocable metadata, concrete routing triggers, fact and concern gates, durable guidance, preserved review/validator ownership, realistic authoring decisions, and hidden installer discovery. All checks passed with a clean worktree.

Evidence: End-to-end decision and discovery transcript
Command:
  bash /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY36W1Y03W4XVP8421HHE5H7/skill-authoring-e2e.sh

Output:
  Skill-authoring standard - end-user decision transcript
  ======================================================
  Scenario 1: Package a current production endpoint and invariant as a skill.
  Decision: REJECT as a skill; route the fact/invariant to AGENTS.md or docs and point to its authority.
  Scenario 2: Combine a release workflow and incident-triage workflow in one skill/PR.
  Decision: SPLIT; one recurring concern per skill and one concern per PR.
  Scenario 3: Draft a repeatable authoring practice with a named before-authoring/substantial-edit trigger.
  Decision: PROCEED; keep principles durable, make the description route the load, and satisfy the seven-item rubric.
  Review safety: the in-draft rubric explicitly preserves independent review-time audit and repository validators.
  Distribution: installer discovery reports only the public stow skill; the new internal standard stays hidden.
  RESULT: PASS

The check also verified:
  - the target range changes only the new standard, the section 13 trigger, and the coding-guidelines cross-pointer;
  - the standard is 73 lines (within the 120-line cap);
  - frontmatter sets user-invocable=false and metadata.internal=true;
  - the description names before-authoring and substantial-edit triggers and distinguishes the generic skill-creator;
  - the shared template contains no Relvino/Ruby/Lylt/Shopify/Shopline/Redpanda/ClickHouse references;
  - live `skills add . --list` discovery finds only the public `stow` skill and does not expose `skill-authoring-standard`.
Evidence: Reproducible end-to-end evidence check
#!/usr/bin/env bash
set -eu

BASE=b4b16035a074e92eb0ac816149635f7d35d7a99d
TARGET=94beac69e7a0108a3325ad8b14984d23f3c07abc
SKILL=.agents/skills/skill-authoring-standard/SKILL.md

fail() {
  printf 'FAIL: %s\n' "$1" >&2
  exit 1
}

[ "$(git rev-parse HEAD)" = "$TARGET" ] || fail "not testing the requested target commit"

expected_files=$(printf '%s\n' \
  .agents/skills/firstmate-coding-guidelines/SKILL.md \
  .agents/skills/skill-authoring-standard/SKILL.md \
  AGENTS.md)
actual_files=$(git diff --name-only "$BASE..$TARGET")
[ "$actual_files" = "$expected_files" ] || fail "target commit range contains unrelated files"

expected_numstat=$(printf '%s\n' \
  $'1\t0\t.agents/skills/firstmate-coding-guidelines/SKILL.md' \
  $'73\t0\t.agents/skills/skill-authoring-standard/SKILL.md' \
  $'1\t0\tAGENTS.md')
actual_numstat=$(git diff --numstat "$BASE..$TARGET")
[ "$actual_numstat" = "$expected_numstat" ] \
  || fail "the trigger/cross-pointer scope is not exactly one line each"

section_13=$(awk '/^## 13\./ {inside=1} /^## 14\./ {inside=0} inside' AGENTS.md)
printf '%s\n' "$section_13" | grep -Fq '`skill-authoring-standard` - load before authoring or substantially editing any skill' \
  || fail "the load trigger is not in AGENTS.md section 13"
grep -Fq 'Load `skill-authoring-standard` before authoring or substantially editing a skill; this skill remains the owner of placement and ownership rules specific to firstmate changes.' \
  .agents/skills/firstmate-coding-guidelines/SKILL.md \
  || fail "the coding-guidelines cross-pointer is absent"

[ "$(wc -l < "$SKILL" | tr -d ' ')" -le 120 ] || fail "standard exceeds 120 lines"

ruby -ryaml -e '
  frontmatter = File.read(ARGV.fetch(0)).split(/^---\s*$\n/).fetch(1)
  data = YAML.safe_load(frontmatter)
  abort "user-invocable is not false" unless data["user-invocable"] == false
  abort "metadata.internal is not true" unless data.dig("metadata", "internal") == true
  description = data.fetch("description")
  abort "description lacks authoring trigger" unless description.include?("before authoring")
  abort "description lacks substantial-edit trigger" unless description.include?("substantially editing")
  abort "description does not distinguish the generic creator" unless description.include?("generic `skill-creator`")
' "$SKILL"

grep -Fq 'If the proposed payload is primarily a domain fact, invariant, concept, source-of-truth statement, or current-state description, stop and place it in the relevant `AGENTS.md` or docs instead.' "$SKILL" \
  || fail "fact/invariant gate is absent"
grep -Fq 'One skill owns one concern, and one PR carries one concern.' "$SKILL" \
  || fail "one-skill/one-PR rule is absent"
grep -Fq 'Teach principles, decisions, and boundaries that survive implementation changes.' "$SKILL" \
  || fail "principle-level durability guidance is absent"
grep -Fq 'Treat the frontmatter `description` as the routing rule because the body is unavailable until the skill loads.' "$SKILL" \
  || fail "description routing rule is absent"
grep -Fq 'This rubric improves construction and does not replace independent review-time audit.' "$SKILL" \
  || fail "review-time audit independence is absent"
grep -Fq 'Leave mechanical syntax, metadata, symlink, and literal-reference enforcement with each repository' "$SKILL" \
  || fail "mechanical validator ownership is absent"

if grep -Eiq 'relvino|ruby-labs|lylt|shopify|shopline|redpanda|clickhouse' "$SKILL"; then
  fail "shared standard contains a project-specific reference"
fi

installer=$(NO_COLOR=1 skills add . --list 2>&1)
printf '%s\n' "$installer" | grep -Fq 'Found 1 skill' || fail "installer did not hide internal skills"
printf '%s\n' "$installer" | grep -Fq 'stow' || fail "public skill disappeared from installer discovery"
if printf '%s\n' "$installer" | grep -Fq 'skill-authoring-standard'; then
  fail "agent-only standard leaked into installer discovery"
fi

printf '%s\n' 'Skill-authoring standard - end-user decision transcript'
printf '%s\n' '======================================================'
printf '%s\n' 'Scenario 1: Package a current production endpoint and invariant as a skill.'
printf '%s\n' 'Decision: REJECT as a skill; route the fact/invariant to AGENTS.md or docs and point to its authority.'
printf '%s\n' 'Scenario 2: Combine a release workflow and incident-triage workflow in one skill/PR.'
printf '%s\n' 'Decision: SPLIT; one recurring concern per skill and one concern per PR.'
printf '%s\n' 'Scenario 3: Draft a repeatable authoring practice with a named before-authoring/substantial-edit trigger.'
printf '%s\n' 'Decision: PROCEED; keep principles durable, make the description route the load, and satisfy the seven-item rubric.'
printf '%s\n' 'Review safety: the in-draft rubric explicitly preserves independent review-time audit and repository validators.'
printf '%s\n' 'Distribution: installer discovery reports only the public stow skill; the new internal standard stays hidden.'
printf '%s\n' 'RESULT: PASS'

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (3) ✅
  • 🚨 AGENTS.md:783 - Intent requires this standard to be “fleet-wide” so “skill crews produce correct first drafts,” but the added trigger only says “load before authoring or substantially editing.” Firstmate delegates project work, while project crewmates do not inherit this repository’s section 13 or internal skills; unlike firstmate-repo briefs, nothing requires project skill briefs to direct crews to this file. Decide whether the trigger and standard should explicitly cover briefing/delegation and require passing the standard to the assigned crew.
  • ⚠️ .agents/skills/skill-authoring-standard/SKILL.md:4 - The description does not follow the new standard’s own requirement to distinguish a plausible neighboring skill. skill-creator matches the same create/update requests, but the routing metadata does not say that both should load or clarify their ownership boundary; that clarification exists only in the body, which is unavailable until routing succeeds. Add the creator/standard boundary to the description if both are intended to load.

🔧 Fix: Clarify skill routing and project crew delegation
1 error still open:

  • 🚨 .agents/skills/skill-authoring-standard/SKILL.md:17 - Intent requires the standard to be “fleet-wide” so “skill crews produce correct first drafts,” but the new instruction passes the relative path .agents/skills/skill-authoring-standard/SKILL.md. Project crews launch with the project worktree as their working directory (for example, the tmux backend uses -c &#34;$proj_abs&#34;), so this resolves inside the project—where this internal Firstmate skill is absent or could name an unrelated file. Pass the resolved absolute path from the active Firstmate home, or another path explicitly rooted at that home, in both this instruction and the section 13 trigger.

🔧 Fix: Resolve delegated skill path from active Firstmate home
1 error still open:

  • 🚨 .agents/skills/skill-authoring-standard/SKILL.md:17 - Intent requires a “fleet-wide” standard, but FM_HOME is explicitly only the operational-state root when set (state/, data/, config/, and projects/); tracked .agents/skills/ remains under the repository root that owns bin/ and AGENTS.md. Thus a supported split-root instance can generate an absolute but nonexistent $FM_HOME/.agents/skills/... path. Resolve the actual repository/instruction root containing this skill (or this file’s own absolute path) rather than directing authors to FM_HOME, and update the duplicated section 13 instruction consistently.

🔧 Fix: Root delegated skill path at Firstmate instruction repository
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"
  • Provided successful baseline: command -v tmux &gt;/dev/null || { echo &#34;tmux is required for e2e tests&#34; &gt;&amp;2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo &#34;== $t ==&#34;; bash &#34;$t&#34; || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit &#34;$rc&#34;
  • Inspected git diff --no-ext-diff --unified=80 b4b16035a074e92eb0ac816149635f7d35d7a99d..94beac69e7a0108a3325ad8b14984d23f3c07abc and compared ownership with the complete generic skill-creator and Firstmate coding guidelines
  • NO_COLOR=1 skills add . --list
  • bash /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY36W1Y03W4XVP8421HHE5H7/skill-authoring-e2e.sh
  • git status --short
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@ruby-dlee

Copy link
Copy Markdown
Owner Author

I reviewed this standard against its own rules. The 73-line internal skill is single-concern and proportionate; its description is a concrete routing rule for authoring and substantial edits and distinguishes the neighboring generic skill-creator. The body keeps facts in docs/AGENTS.md, requires principle-level content and one concern per skill/PR, and points to neighboring owners instead of copying their contracts. The only companion edits are the minimum section-13 trigger and one reciprocal coding-guidelines pointer, including the project-crew briefing path boundary. I found no fleet-specific leakage or merge blocker.

Verdict: ready.

@ruby-dlee
ruby-dlee merged commit 6be4c7c into main Jul 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant