test(scaffold): add frontmatter spec gate for skills and agents#55
Merged
shyamsridhar123 merged 5 commits intoJun 7, 2026
Merged
Conversation
Add skillspec_test.go, a structural regression guard that validates the frontmatter of every shipped Copilot definition under .github/skills, .github/agents, and pkg/scaffold/templates/skills stays well-formed and complete, so definitions cannot silently drift out of spec as the set grows. Calibrated to existing conventions so it is green on current main: requires well-formed standalone --- delimiters, required keys (skill name/description, agent description/user-invocable), name == folder for skills, unique skill names within a tree, no duplicate keys, non-placeholder descriptions, and a non-empty body. It deliberately does NOT enforce a description length cap, required body sections, or any naming style beyond name == folder. Includes unit tests proving each defect class is rejected (not a vacuous gate). pkg/scaffold/templates/agents is intentionally excluded: those files are currently stored single-line (collapsed frontmatter) and are not standard line-delimited YAML frontmatter; tracked as follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… chore/skill-spec-gate
Now that the scaffolded agent templates are repaired (no longer single-line minified), hold pkg/scaffold/templates/agents to the same structural contract as .github/agents: well-formed delimited frontmatter, a non-placeholder description, a boolean user-invocable, and a non-empty body. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
shyamsridhar123
added a commit
that referenced
this pull request
Jun 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a structural regression gate (
pkg/scaffold/skillspec_test.go) thatvalidates the YAML frontmatter of every shipped Copilot skill and agent
definition, so the repo's growing definition set cannot silently drift out of
the shape the Copilot harness expects.
Ported from the governance
verify.shspec-checks incopilot-agents-dojo,
re-expressed as a native Go test so it runs inside the existing
go test ./...CI with zero new tooling.
Trees covered
.github/skillsand.github/agents(canonical dogfood definitions)pkg/scaffold/templates/skillsandpkg/scaffold/templates/agents(consumer-facing templates)
Invariants enforced (calibrated to pass green on current
main)---/---delimiters (CRLF-normalized, so.githubCRLF andtemplate LF are treated identically)
name+description; agentdescription+user-invocablenamematches its folder and is unique across a treeTODO/TBD/…)user-invocable∈ {true,false}Deliberately no stylistic policy ATV doesn't already follow (no description
length cap, no required-section list, no naming-style rule beyond
name==folder).A self-test (
TestParseFrontmatterRejectsMalformed) proves the gate is notvacuous.
Stacked on #56
The agent-template coverage depends on the corruption repair in #56. This
branch therefore merges #56, so until #56 lands this PR's diff temporarily
includes #56's files. Review/merge #56 first; this diff will then reduce to the
test file (+ CHANGELOG).
Verification
go test ./pkg/scaffold/passes. (The only failure in this Windows env,TestDogfoodPromptParity, is a pre-existing CRLF artifact unrelated to thischange and green on Linux CI.)