feat(copilot): add a pull-request integrity check for GitHub Copilot - #68
Merged
Conversation
Copilot's composition lives in the repository. Its instructions, skills and MCP configuration are files that arrive by pull request, which makes a different control correct here than for the other engines. The Claude Code and Codex integrations watch a developer's machine and warn at session start, after the fact, one developer at a time. They have to: that composition lives in a home directory. Here the composition is reviewed code, so drift is caught on entry. One baseline committed at .agentrust/copilot-baseline.json, and a status check that fails a pull request changing what Copilot reads without updating the baseline in the same change. Paths were verified against GitHub's documentation, and the surface is wider than expected. Copilot reads AGENTS.md ANYWHERE in the tree, nearest wins, plus root CLAUDE.md and GEMINI.md as alternatives, plus .github/copilot-instructions.md and .github/instructions/**/*.instructions.md. Skills resolve from three in-repo roots: .github/skills, .claude/skills and .agents/skills. So a file three directories down changes how the agent behaves in that subtree without touching anything at the root, which is exactly the change worth catching. Vendored directories are skipped so a dependency shipping its own AGENTS.md is not counted as ours. Skills are digested across the whole directory via the shared core, so the bypass that was live in two other engines does not reappear here. Copilot skills use the same SKILL.md plus supporting-files shape, so it would have. This engine deliberately does NOT seal its baseline, unlike the others. They seal because a local baseline can be rewritten with nothing to show for it. A committed baseline gets provenance from git: every change appears in a diff, carries an author, and passes review. A digest on top would be ceremony. The action needs no install step, since the engine and its vendored core are standard library only. fail-on-drift defaults true but can be turned off, which is the sensible first move on a busy repository. The comment is one per pull request, edited in place, because a comment per push is noise people mute. A missing baseline reports and exits 0 rather than blocking a repository that has not adopted one. No integration.yaml. The schema requires integrates_with to be one of cmcp, trace or agent-manifest, and this check emits none of them, so claiming one would be an unverifiable claim and CONTRIBUTING is explicit about those. The README says so and names emitting a TRACE record per checked pull request as what would make one true. 25 tests. The suite imports its engine by path under a unique module name: four engines here each define a module called `capture`, so sys.path insertion made this file run against another engine's code when the repository was collected in one command. Also named test_copilot_capture.py rather than test_capture.py so it does not join the pre-existing basename collision between the other suites, which still breaks a single root-level pytest run and is worth a follow-up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
#67 added the mode parameter to core.atomic_write while this branch was open, so the copilot vendored copy was a version behind. Caught by the vendored-in-sync job against the merge with main, which is exactly what that check is for. Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
This was referenced Aug 1, 2026
4 tasks
imran-siddique
pushed a commit
that referenced
this pull request
Aug 19, 2026
* feat: add agent surfaces for Cursor, Windsurf, Gemini CLI Three new pull-request drift checks, the same shape #68 established for Copilot: rules/context, skills and MCP configuration are files that arrive by pull request, so each is a status check rather than a session hook. Each path list was verified against current vendor documentation rather than assumed from the issue's own table, per #78's own instruction, and that changed the design in both directions: - Cursor: .cursor/rules/*.mdc is measured one level deep only, since Cursor's own forum documents nested rule files as not reliably read. Skills, by contrast, are measured anywhere in the tree, since Cursor's docs describe monorepo-scoped skill roots as intentional. - Windsurf: caught mid-rebrand from Cognition's Devin acquisition. .devin/rules/ is now preferred over .windsurf/rules/, verified directly against docs.windsurf.com's redirect to docs.devin.ai, but the skills surface has not moved the same way as of this writing, verified separately rather than assumed. Windsurf has no repository-resident MCP surface at all (home-directory only), so that engine ships with no mcp category. - Gemini CLI: GEMINI.md is measured anywhere in the tree per the documented context hierarchy; MCP configuration lives in .gemini/settings.json, digested whole rather than parsed for the one key that matters, the same tradeoff Copilot's engine makes for devcontainer.json. All three share agentrust-capture-core, wire up tests + integrity CI workflows matching the Copilot pattern's pinned action SHAs, and ship empty approved baselines in this repo for self-check dogfooding. 152 tests passing across all engines in this repo; no collisions with existing capture-core consumers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com> * fix(cursor): recurse .cursor/rules and measure AGENTS.md Direct confirmation against cursor.com/docs (Customize > Rules, Customize > MCP) corrected two things the earlier forum/blog-sourced research got wrong: - .cursor/rules/*.mdc was globbed one level only, based on a forum report that nested rule files were unreliable. The official docs show nested folders as an intended pattern (.cursor/rules/frontend/components.mdc, presented as normal organisation, not an edge case), so a real nested rule file was silently not tracked for drift. Now globs recursively. - AGENTS.md was missing entirely. It's one of exactly four documented Cursor rule types ("a simple alternative to .cursor/rules"), read from the project root and subdirectories, with "Nested AGENTS.md support" as a shipped improvement. Now measured anywhere in the tree, same as Copilot's engine already does for the same file. .cursorrules and the .cursor/mcp.json / ~/.cursor/mcp.json split are both confirmed accurate as originally shipped; only the citations changed from secondary-source hedging to direct doc references. .cursorrules itself does not appear in current docs at all (four rule types are listed and it isn't one of them), so its README note now says that plainly rather than repeating an unconfirmed forum deprecation claim. 28 tests now (was 24): the flipped nested-mdc assertion plus four new AGENTS.md/plain-.md-extension cases. Full repo suite still green (156 tests across all five engines), self-check baseline still passes clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com> * docs(cursor): confirm skills paths directly against cursor.com/docs The skills implementation (root-anywhere-in-tree, recursive category subfolders, the four skill roots) turned out to already match Cursor's official docs (Customize > Skills) exactly, unlike rules and AGENTS.md in the previous commit. No logic change here, just upgrading the citations from secondary-source hedging to a direct doc reference, and adding one honest gap to the README: Cursor's built-in skills (/automate, /babysit, etc.) ship with the product itself, not as repository files, so they're correctly out of scope for a file-based check. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com> * docs(cursor): trim the .cursorrules note to what's actually confirmed The previous wording cited "no staff confirmation" from Cursor's community forum as if the forum had been checked broadly. Only one thread was actually checked, once. Trimmed the claim to what's verifiable: .cursorrules is absent from current docs, and whether Cursor still reads it is unconfirmed either way, full stop. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com> * docs(cursor): reflect community consensus that .cursorrules still works Additional research (independent of the earlier single forum thread) turned up several converging community sources describing .cursorrules as still read today, applied globally, just deprioritised in favour of .cursor/rules .mdc files. Still not confirmed by any official Cursor source, so the note says exactly that rather than treating it as settled. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com> --------- Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fourth engine, and the first one that is not a session hook.
Why the shape is different
Copilot's composition lives in the repository. Its instructions, skills and MCP configuration are files that arrive by pull request.
The Claude Code and Codex integrations watch a developer's machine and warn at session start, after the fact, one developer at a time. They have to: that composition lives in a home directory. Here it is reviewed code, so drift can be caught on entry:
.agentrust/copilot-baseline.json, not one per laptop.CODEOWNERS for the agent's brain, roughly.
The surface is wider than I expected
I said I would verify the paths against GitHub's docs before building, and it changed the design:
.github/copilot-instructions.md,.github/instructions/**/*.instructions.md,AGENTS.mdanywhere in the tree, rootCLAUDE.mdandGEMINI.md.github/skills/<n>/,.claude/skills/<n>/,.agents/skills/<n>/copilot/mcp-config.json,.vscode/mcp.jsonAGENTS.mdis matched anywhere, because Copilot resolves the nearest one. A file three directories down changes how the agent behaves in that subtree without touching anything at the root, and that is precisely the change worth catching. Vendored directories are skipped, so a dependency shipping its ownAGENTS.mdis not counted as yours. Tested both ways.Skills are digested across the whole directory via the shared core, so the bypass that was live in two other engines (#63, #65) does not reappear. Copilot skills use the same
SKILL.mdplus supporting-files shape, so it would have.Two deliberate omissions, both load-bearing
No baseline sealing, unlike the other three. They seal because a local baseline can be rewritten with nothing to show for it. A committed baseline gets provenance from git: every change appears in a diff, carries an author, passes review. A digest on top would be ceremony, and shipping ceremony as security is the habit this repo argues against.
No
integration.yaml. The schema requiresintegrates_withto be one ofcmcp,traceoragent-manifest. This check emits none of them, so claiming one would be an unverifiable claim, andCONTRIBUTINGrule 2 is explicit about those. The README says so plainly and names emitting a TRACE record per checked pull request as what would make one true. That is also the natural first consumer fortrace-registry.Worth noting the schema assumes every integration integrates with the AgenTrust stack. A drift checker that uses the shared core but emits no record has no honest value to put there, which may be a gap in the schema rather than in this integration.
Adoption details that matter more than they look
fail-on-driftcan be turned off. Report without blocking is the sensible first move on a busy repo; flip it on once the baseline settles.Test plan
.github/instructions,AGENTS.mdat depth, vendoredAGENTS.mdexcluded, the skill-script bypass, state churn not alarming, both MCP paths, and verify's exit codes as a status check.snapshotandverifyrun against this repository in CI. It currently has none of these surfaces, so the counts are zero andverifysays there is no baseline. Honest, and it becomes a real gate on ourselves once one is committed.ruff check --target-version py39clean. Vendored sync check passes across all four engines.One repo-wide problem I hit and worked around. Four engines each define a module named
capture, sosys.pathinsertion made this suite run against another engine's code when the repository is collected in one pytest command: 25 failures against the wrong module. This suite now imports its engine by path under a unique name, and is namedtest_copilot_capture.pyso it does not join the pre-existingtest_capture.pybasename collision between the other three. A single root-levelpyteststill fails on that older collision, which predates this PR and deserves a follow-up.