Skip to content

spec-reference.md

Latest commit

 

History

History
503 lines (399 loc) · 21.1 KB

spec-reference.md

File metadata and controls

503 lines (399 loc) · 21.1 KB
Raw

Spec reference — .skillgate/done.yaml

A spec is a YAML (or JSON) file. skillgate looks for it at, in order: .skillgate/done.yaml, .skillgate/done.yml, .skillgate.yaml, .skillgate.yml, .skillgate.json. A machine-readable JSON Schema lives at schema/done.schema.json.

Top-level fields

Field Type Required Meaning
gates array yes The deterministic checks. Order is preserved in output.
finishLine string[] no Command prefixes that cross the finish line, matched structurally by agent integrations.
name string no A label for this definition of done.
version integer no Spec format version. Omit for the current format. See compatibility.
timeout positive integer no Total wall-clock budget for one run in milliseconds. Defaults to 300000.
gatedTools string[] no Agent tool names (globs) that also cross the finish line, such as MCP tools. See gated tools.

finishLine uses structural shell matching. Each unquoted command segment is tokenized, common wrappers and executable suffixes are normalized, and the command prefix is compared with the configured pattern. This catches env CI=1 git push, git -C repo commit, nested sh -c/PowerShell commands, and Windows .cmd launchers without matching quoted prose. Run skillgate explain --command "..." to inspect the decision without executing gates.

Runtime loading enforces the same closed shape as the JSON Schema: unknown fields, unsupported gate types, duplicate IDs, invalid regexes, invalid option values, and empty gate lists are rejected before any gate runs. A configured but invalid policy blocks agent hooks rather than being ignored.

Gate types

trufflehog adds optional verified credential scanning and a private literal-name denylist. review consumes a completed report bound to the current working/staged snapshot. Their fields, setup, and trust limits are documented in secrets and optional review.

Every gate has an id (string, shown in output), an optional description, and an optional when block.

file-exists

Every listed path must exist.

- id: docs
  type: file-exists
  file: [README.md, LICENSE]   # string or array of strings

file-contains

A file must contain a regex match — a required section, an exact phrase, a version bump.

- id: changelog-touched
  type: file-contains
  file: CHANGELOG.md
  pattern: "unreleased"
  flags: "i"                   # optional JS regex flags

absent

A regex must not appear in any matched file. Reports the first file:line hit. Ideal for stray TODOs and committed secrets.

- id: no-secrets
  type: absent
  glob: "**/*.{ts,js,json,md,yaml,yml,env}"
  pattern: 'sk_live_[A-Za-z0-9]{16,}'
  ignore: [".skillgate/**"]    # optional extra excludes

node_modules/, .git/, and dist/ are always ignored.

A glob that matches no files fails the gate: a typo'd glob would otherwise pass forever as a no-op. Set allowEmpty: true when an empty match is expected. The same rule applies to no-new, no-fewer and no-deleted.

Pattern gates (file-contains, absent, no-new, no-fewer) read at most maxBytes per file (default 10 MiB). A larger file fails the gate with its size rather than being skipped. Exclude it with ignore or raise maxBytes. They also stop at the run's timeout and report how many files they scanned.

command

A shell command must exit 0. Only as deterministic as the command itself — prefer test/lint/build commands, not anything that hits the network.

- id: tests-pass
  type: command
  run: "npm test --silent"
  timeout: 60000               # optional, default 30000 (30s)

On timeout the gate returns a deterministic command timed out after Nms reason.

When a base ref resolves, the command also receives the files changed against it:

Variable Value
SKILLGATE_CHANGED_FILES path of a temporary file listing the changed files, one per line, relative to the policy's directory
SKILLGATE_CHANGED_COUNT how many files the list holds

The list covers committed, staged, unstaged and untracked changes, leaves out deleted files, and keeps only files matching the gate's when.changed globs when it has them. Without a base both variables are unset, so a command can fall back to a full run instead of checking nothing:

- id: lint-changed
  type: command
  run: 'if [ -n "$SKILLGATE_CHANGED_FILES" ]; then xargs -r npx eslint < "$SKILLGATE_CHANGED_FILES"; else npx eslint .; fi'
  when:
    changed: ["src/**/*.ts"]

trivy

Run Trivy as a first-class security gate. By default this blocks on any leaked secret, any CRITICAL vulnerability, or failure to generate a CycloneDX SBOM.

- id: trivy-clean
  type: trivy
  target: "."                 # optional, default "."
  scanners: ["vuln", "secret"] # optional, default shown
  severity: ["CRITICAL"]       # optional, vulnerability scan only
  sbom: true                   # optional, default true
  timeout: 120000              # optional, per Trivy invocation

The secret scan runs separately from the vulnerability scan, so CVE severity filtering does not hide leaked credentials. After a passing vulnerability scan, one more JSON scan counts the findings below the blocking severities and the gate reports them (not blocking: 12 HIGH, 3 MEDIUM). The count is informational and never changes the verdict; set summary: false to skip it. Set trivy when the binary is not on PATH, or ignoreUnfixed: true when the vulnerability policy should ignore unfixed CVEs.

evidence

The escape hatch for steps that are not machine-observable ("research X first"): the agent writes a named file as it works, and the gate verifies it is a non-empty regular file. Directories and unreadable paths fail the check.

- id: research-recorded
  type: evidence
  file: .skillgate/evidence/research.md

not-empty

A directory must contain at least min entries (files or subdirectories). Catches agents that claim a step is done but leave an empty evidence/, docs/ or dist/ directory.

- id: evidence-not-empty
  type: not-empty
  path: docs/api
  min: 3                    # optional, default 1

instruction-sync

Every AI instruction file in the repo (CLAUDE.md, AGENTS.md, .cursor/rules, copilot-instructions.md, …) must still agree with the canonical one. Drift means your agents are reading different rulebooks. Run skillgate sync to fix, or skillgate diff-instructions to see exactly what changed.

- id: instructions-in-sync
  type: instruction-sync
  threshold: 0.95              # optional, 0..1, default 0.95
  require: [claude-code, gemini-cli]   # optional

require names tools that must have their own instruction file, in sync with or linked to the canonical one. Without it, a repository with only AGENTS.md passes, yet some tools read AGENTS.md only through a fallback that can be switched off (Claude Code skips it when, for example, only a personal CLAUDE.local.md is present). Tool ids: agents-md, claude-code, cursor, github-copilot, gemini-cli, cline, windsurf, jetbrains-junie. skillgate sync --create claude-code,gemini-cli writes the missing files as @AGENTS.md pointers (or synced copies for tools without import support).

instruction-refs

Every repository path an instruction file points at must exist: @imports (@docs/rules.md), relative Markdown links, and code spans that name a directory-qualified path (src/cli.ts, docs/). Rules about files that are gone send the agent after a repository that no longer exists.

- id: instruction-refs
  type: instruction-refs
  ignore: ["dist/**"]          # optional: referenced paths to skip

Fenced code blocks, URLs, absolute and home paths, and bare file names are not judged. A slashed code span counts as a path only when it has a file extension, ends with /, or its first segment exists, so origin/main and owner/repo are left alone. References resolve against the instruction file's directory or the repository root. Failures point at file:line.

no-new, no-fewer, no-deleted (diff-aware)

These gates compare the working tree with the commit the change forked from (--base <ref>, SKILLGATE_BASE, or origin's default branch). Without a resolvable base they fail closed. The one exception is a repository with no commits at all: its first commit is judged against the empty tree.

An explicit --base or SKILLGATE_BASE must resolve; Skillgate never replaces a misspelled request with another branch. Unreadable baseline trees fail the gate, and unrelated histories cannot supply a common ancestor. Globs use the same syntax for on-disk and historical paths, including braces, character classes, and extglobs. Historical reads are scoped to the policy's workspace.

- id: no-new-skips               # count must not increase
  type: no-new
  glob: "**/*.test.ts"
  pattern: '\.(skip|only)\('

- id: tests-kept                 # count must not decrease
  type: no-fewer
  glob: "**/*.test.ts"
  pattern: '^\s*(it|test)\('

- id: tests-not-deleted          # every file at the base must still exist
  type: no-deleted
  glob: "test/**"

no-fewer catches a suite made green by deleting test cases inside files that still exist, which no-deleted (whole files) and no-new (added skips) miss.

unchanged (diff-aware)

Every file matching glob that existed at the base must still exist with identical content. Use it for the files that define "correct": snapshots and golden outputs, migrations that already ran, CI workflows, lint configuration. An agent that edits them to make a check pass is blocked; new files are allowed.

- id: protected
  type: unchanged
  glob: "{**/__snapshots__/**,migrations/**,.github/workflows/**}"
  ignore: ["migrations/README.md"]   # optional

Content is compared the way Git would store it, so line-ending normalization from .gitattributes does not count as a change. Like the other diff-aware gates it fails closed without a base. Change a protected file in a separate, reviewed commit, or pin the policy with --pin.

signed-commits (diff-aware)

Every commit between the base and HEAD must be signed. Scope it to the push:

- id: signed
  type: signed-commits
  trust: signed                # optional: signed (default) or verified
  when:
    command: ["git push"]

signed accepts any signature Git did not reject, including one whose key is not in the local keyring. verified requires a good signature Git can check (GPG trust, or gpg.ssh.allowedSignersFile for SSH signing). Unsigned commits, bad signatures and revoked keys always fail. No commits since the base passes.

deps-declared

Every package a JavaScript or TypeScript file imports must be declared in the nearest package.json (dependencies, devDependencies, optionalDependencies or peerDependencies). An import that works only because another package hoisted it, or because it is installed globally, fails for the next user; this catches it offline. Pair it with deps-locked, which checks the manifest against the lockfile.

- id: deps-declared
  type: deps-declared
  glob: "src/**/*.{ts,tsx,js,mjs,cjs}"
  allow: ["@/*", "virtual:*"]  # optional: aliases treated as declared

import … from, export … from, side-effect imports, require() and dynamic import() with a literal specifier are read; comments are ignored. Relative paths, #subpath imports, protocol specifiers (node:, bun:) and Node builtins are skipped, as are a package's imports of itself. A type-only import is also satisfied by @types/<name>. Workspace packages answer to their own manifest.

deps-locked

Every dependency declared in a manifest must be present in its lockfile. A package that never resolved from a registry, such as a hallucinated name, cannot be in the lockfile, so this catches it offline and deterministically.

- id: deps-locked
  type: deps-locked
  manifest: package.json        # optional; default: every supported manifest at the root

Supported: package.json with package-lock.json, npm-shrinkwrap.json, pnpm-lock.yaml, yarn.lock or bun.lock (dependencies, devDependencies and optionalDependencies; file:/link: specs are skipped), and pyproject.toml ([project] dependencies, optional dependencies, [dependency-groups], and all Poetry dependency groups) with uv.lock, poetry.lock or pdm.lock. Python manifests and lockfiles are parsed as TOML, so comments and unrelated metadata cannot supply package names. Dependency-group includes resolve normalized names; unknown groups, cycles, malformed entries and duplicate normalized group names fail closed.

For pnpm, only the manifest's importer can satisfy its dependencies. Nested manifests can use the nearest ancestor's shared pnpm-lock.yaml; importer keys are relative to the lockfile directory. Lookup stops at a nested pnpm workspace boundary without a lockfile. Adjacent lockfiles retain precedence, and legacy flat pnpm locks remain supported for the root manifest. Receipts include the selected shared lockfile even when it sits outside the policy directory. No manifest or no lockfile fails the gate.

phase

Phases run in order (plan → build → review, say). To be in a phase, every gate listed in requires by that phase and by every earlier phase must pass right now. The gate reads the active phase from current, a plain-text file (default .skillgate/phase); no file means the first phase.

- id: plan-written
  type: file-exists
  file: docs/plan.md
- id: tests-pass
  type: command
  run: npm test --silent
- id: phases
  type: phase
  phases:
    - id: plan
    - id: build
      requires: [plan-written]
    - id: review
      requires: [tests-pass]

Nothing is recorded or signed: the marker only says where the agent claims to be, and every evaluation re-checks that claim against the workspace. Writing review into the marker without a plan or passing tests just makes the gate fail at the next finish line or gated tool. requires may name any non-phase gate in the spec; each required gate runs once per run.

Move between phases with skillgate phase <id>: it writes the marker only if the target's requirements pass (exit 2 otherwise). skillgate phase alone shows each phase's status.

Gated tools

finishLine gates shell commands. gatedTools gates any other agent tool by name, typically MCP tools that publish, deploy or send:

gatedTools: ["mcp__course__publish_*"]

A call to a matching tool runs the gates like a finish-line command. Scope a gate to tools with when.tool (below). Hooks from skillgate install claude-code, codex and gemini-cli include these tools in their matcher, and doctor reports a hook that does not. The opencode plugin checks gatedTools in tool.execute.before. Cursor's hook covers shell commands only.

Conditional gates (when)

when limits where a gate applies. Every listed condition must hold. Within a list, any entry may match. A gate whose condition does not hold is reported as skipped and never blocks.

- id: full-suite
  type: command
  run: npm run test:all
  when:
    command: ["git push", "npm publish"]  # required for push/publish, not every commit
    tool: ["mcp__course__publish_*"]      # and for these gated tools
    changed: ["src/**", "package.json"]   # only if one of these changed vs the base
    branch: ["main", "release/*"]         # only on these branches
  • command and tool scope a gate to actions: when a command or a gated tool is being judged, the gate applies only if it is listed in one of them.
  • command is matched structurally like finishLine. skillgate gate judges the agent's command. skillgate check --command "git push" evaluates as if for that command. A plain check has no command, so every gate applies.
  • changed compares committed, staged, unstaged and untracked files against the base ref.
  • branch uses SKILLGATE_BRANCH, else the checked-out branch, else the CI branch (GITHUB_HEAD_REF, GITHUB_REF_NAME).

A condition skillgate cannot decide (no base ref, no branch) runs the gate. An unknown never skips enforcement. Pin the policy with --pin so a change cannot add a when that exempts itself.

skillgate init now defaults

Starting with v0.5.0, skillgate init generates a template that includes the instruction-sync gate and a commented-out evidence gate example. Every new project starts with drift detection enabled and an evidence workflow ready to activate — no opt-in required.

Scaffold templates (skillgate scaffold)

The skillgate scaffold command generates .skillgate/evidence/ with expected files for the agent to write before crossing the finish line. It also creates a README explaining the evidence workflow.

skillgate scaffold                          # generic evidence files
skillgate scaffold --template react         # React / Next.js stack
skillgate scaffold --template ts-lib        # TypeScript library
skillgate scaffold --template python        # Python application
skillgate scaffold --update-agents          # also update AGENTS.md/CLAUDE.md

Each template generates:

  • test-output.txt — save test runner output here
  • lint-report.txt — save linter output here
  • diff-review.md — self-review of changes
  • README.md — explains the evidence workflow to the agent

Stack-specific templates add files like typecheck-output.txt (ts-lib, react, python) or coverage-summary.txt (ts-lib).

--update-agents

When passed, skillgate scaffold also appends (or creates) workflow instructions in AGENTS.md (or CLAUDE.md if AGENTS.md is absent) that tell the agent to:

  1. Run checks and save output to the evidence files
  2. Write a self-review of changes
  3. Run npx skillgate check before crossing the finish line

Determinism contract

Every gate is a pure function over the filesystem: same inputs, same verdict, in milliseconds, with no model in the loop. A command gate inherits the determinism of the command you give it — keep them hermetic.

Policy discovery starts in the requested directory and walks upward only to the current Git worktree root. Gates run relative to the directory that owns the policy, so calling skillgate check from a nested package cannot accidentally use paths from the main checkout or a parent repository.

The run budget applies across all gates and cumulative phase requirements. A per-command timeout is capped by the remaining total budget, and a timed-out command's supervised process tree is terminated. Any gates that could not start are returned as blocking not-run results, preserving one result per configured gate. Commands have a combined 8 MiB output cap. On Unix, completion also terminates background descendants in the shell's process group; command gates should finish their work before exiting. Windows timeout cleanup uses taskkill /T when available.

skillgate check --receipt <file> writes a versioned JSON receipt with the workspace snapshot, per-gate status/reason/duration, and total budget. --cache stores a passing receipt outside the working tree and reuses it only for the same parsed policy, runtime, base commit, branch/index state, and repository snapshot, including explicit ignored gate inputs and symlink file contents. Failing or malformed results are never reused. A workspace change during evaluation blocks a passing receipt. Receipt outputs must not overlap gate input paths or globs.

Cache reuse applies to local file, directory, pattern, diff (including unchanged), and phase checks. Commands, Trivy, TruffleHog, reviews, instruction selection and references, signed commits, and dependency checks always execute again because their complete inputs are not represented by the snapshot. With --json, cacheDisabledReason explains a requested cache bypass.

Decision log

--log <file> on check and gate, or SKILLGATE_LOG=<file>, appends every verdict as one JSON line: time, via (gate or check), event (command, tool, stop, check), decision, reason, the judged command or tool, the failed gate ids with their reasons, and the duration. Agent hooks call gate for every shell command, so the log also records the commands that were allowed.

Keep the log outside the worktree, or under .git/ (.git/skillgate/decisions.jsonl in a regular checkout). A path inside the worktree would change the snapshot the gates judge, so it is refused. Logging never changes a verdict: a log that cannot be written is reported on stderr and skipped.

skillgate log .git/skillgate/decisions.jsonl          # blocks per gate, blocked commands
skillgate log --json                                   # same, from SKILLGATE_LOG
skillgate explain --commands .git/skillgate/decisions.jsonl   # replay against today's finishLine

explain --commands <file> reads one command per line (# comments allowed) or decision-log records, and reports which commands cross the finish line, which pattern matched, and which gates their when.command scope would run. Pass - to read stdin. Use it before you change finishLine or a when block.