Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,23 @@

## Unreleased

## 0.12.0 - 2026-10-01

### Added
- `init --preset no-secrets` and optional `trufflehog` gate with verified credential
detection, a private literal infrastructure-name denylist, staged-file checks,
bounded execution, and redacted output. TruffleHog remains an external optional binary.
- Optional `review` report gate and `review-snapshot` command for OCR-managed or
delegated reviews. Reports must complete without findings or warnings and match
the working/staged snapshot. This is explicitly a trusted reviewer attestation.
- Processbench-style real-PR regression evidence and external-gate setup documentation.

### Fixed
- Invalid opencode policies now block unknown/MCP publish tools while keeping
built-in file-repair tools available.
- External gates always re-evaluate with `--cache`; phase evaluation carries the
complete policy when checking a review report.

### Documentation
- Added a Worktrunk pre-merge hook recipe using the existing worktree-local policy
discovery, with an explicit merge target for diff-aware gates.
Expand Down
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,11 @@ Prompt-level fixes ("always follow the process") do not close the gap, because t

![How skillgate works: an AI agent tries to commit, skillgate runs deterministic gates outside the model, and blocks the finish line until every gate passes](assets/skillgate-flow.png)

The check is a **pure function over the filesystem**: same inputs, same verdict, in milliseconds, with no model in the loop. That is the whole point. An LLM asked "is this done?" answers differently depending on the weather and has an incentive to say yes. A script does not. Because the judge is model-independent, it works the same whatever model you have plugged into your agent.
The default filesystem checks produce the same verdict for the same inputs, with
no model in the evaluator. Commands and optional credential verification may
depend on external state. The optional review gate checks a report supplied by a
trusted reviewer; it is an attestation with a different trust boundary. The gate
engine remains independent of the model plugged into your agent.

## Gate, not loop

Expand Down Expand Up @@ -174,6 +178,11 @@ open. Re-running `install` upgrades it in place.

## Define your gates

Use `skillgate init --preset no-secrets` for credential and private infrastructure
checks. An optional `review` gate accepts a snapshot-bound OCR or host-agent
report. See [secrets and optional review](docs/external-gates.md) for setup, limits,
and reviewer trust. Both integrations are optional.

A gate is one deterministic, machine-checkable condition. Run `npx @reneza/skillgate init` to drop a starter `.skillgate/done.yaml` that includes drift detection and an evidence-gate example right out of the box, then run `skillgate scaffold` to generate the evidence file templates the agent must fill in:

```bash
Expand Down
115 changes: 115 additions & 0 deletions docs/external-gates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Secrets and Optional Code Review

## No-secrets preset

Install [TruffleHog](https://github.com/trufflesecurity/trufflehog), then initialize:

```sh
skillgate init --preset no-secrets
skillgate check
```

The preset refuses to overwrite an existing policy. For an existing policy, add:

```yaml
- id: no-secrets
type: trufflehog
timeout: 10000
namesFile: .skillgate/forbidden-names.json
```

Set the local denylist to a non-empty JSON array of literal server, domain, or
tenant names. Initialization starts with `[]` and blocks until you configure it.
Remove `namesFile` for credential scanning alone. The preset ignores the private
file and scanner clone directory. Provide the denylist separately to trusted CI.
An existing policy needs these ignore entries added explicitly:

```gitignore
/.skillgate/forbidden-names.json
/.skillgate/scan-cache/
```

The name check uses case-insensitive literal matching over tracked and unignored
untracked files and the staged index. Missing, malformed, or empty configuration,
symlinks, unreadable files, and size/time limits block. Matched names are redacted.

The credential check runs `trufflehog git file://<repo> --results=verified --fail
--fail-on-scan-errors --no-update --json` with temporary clones under the ignored
scan directory. The executable is optional and never downloaded by Skillgate.
Set `trufflehog` to override its path. A missing scanner, nonzero exit, malformed
output, or timeout blocks. Raw output never enters receipts. Policies containing
this gate always re-evaluate, including with `--cache`.

Verified-only detection requires network access to credential providers. It does
not guarantee detection of fake, expired, unsupported, or unverifiable credentials.
Keep static `absent` gates where those are also forbidden. Verification responses
and scan times can change with identical files: this optional gate is not a pure
offline filesystem predicate.

TruffleHog v3.97.9 took 3.7 seconds on a small TypeScript repository. The vendor's
public AWS/URI canaries triggered exit 183 in history and when staged before
commit. Published evidence contains no credential values. Measure your repository:
ten seconds is a budget, not a universal performance guarantee.

## Optional OCR Review Adapter

A review gate consumes a trusted reviewer's report. It does not run a model or
add [Open Code Review](https://github.com/alibaba/open-code-review) as a dependency:

```yaml
- id: review
type: review
file: .skillgate/review.json
```

Ignore `.skillgate/review*` before taking a snapshot. Capture `skillgate
review-snapshot` **before** reviewing and retain that value in the report. The hash
covers policy, working files, and staged blob IDs. Changes require another review.
Use the same `--base`/`--pin` options for snapshot and evaluation. `--cache` never
skips this gate.

OCR v1.12.11 has two paths:

```sh
# OCR-managed model; a custom/local endpoint is also possible.
ocr review --format json --output .skillgate/review.raw.json

# Host-agent review; no separate OCR API key or model configuration.
ocr delegate preview --format json
ocr delegate rule src/core.ts src/plugin.ts --format json
```

Delegation prepares file selection and rules. The host must inspect the actual
diff and complete the review. `delegate preview` alone is not a completed review.

Wrap OCR JSON, or a delegated review using the same status/summary/comments shape,
in this envelope. Normalize absent `warnings` to an empty array:

```json
{
"format": 1,
"snapshot": "<value captured before review>",
"review": {
"status": "success",
"summary": { "files_reviewed": 2, "comments": 0 },
"comments": [],
"warnings": []
}
}
```

Missing, stale, incomplete, skipped, budget-exceeded, or malformed reports block.
Any comment or warning blocks. Resolve findings and rerun review. Comment content
is never printed by this gate.

This is a **reviewer attestation**, not independent proof that review ran or that
code is correct. An agent with report-write access can forge it. Put policy and
report production behind a trusted boundary for independent enforcement.

The no-key thin slice used [Skillgate PR #39](https://github.com/renezander030/skillgate/pull/39).
OCR selected six source/schema files and excluded documentation and tests. The
host review identified an invalid-policy path allowing non-shell publish tools
through the opencode plugin. A regression test reproduced it; the fix blocks
unknown/MCP tools on invalid policy while allowing built-in repair tools. This is
one host-reviewed case, not a model-quality comparison. Review documentation and
tests separately when OCR excludes them.
5 changes: 5 additions & 0 deletions docs/spec-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ blocks agent hooks rather than being ignored.

## Gate types

`trufflehog` adds optional verified credential scanning and a private literal-name
denylist. `review` consumes a completed report bound to the current working/staged
snapshot. Their fields, setup, and trust limits are documented in
[secrets and optional review](external-gates.md).

Every gate has an `id` (string, shown in output), an optional `description`, and
an optional [`when`](#conditional-gates-when) block.

Expand Down
1 change: 1 addition & 0 deletions examples/review-regressions.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id":"review-invalid-policy-publish","workflow":"agent-coding-approval-boundary","input":"Inspect Skillgate PR 39 at cbbd8e99fe19236297b78c052a9c685b54900d93. A malformed policy must not permit a non-shell publish tool through the opencode plugin.","expected":{"change_type":"invalid-policy-fail-closed","publish_blocked":true,"repair_tools_available":true},"hard_blockers":["publish_allowed_on_invalid_policy"],"observed":{"existing_ci":"pass","host_delegated_review":"finding","new_regression_test":"pass_after_fix"},"source":"https://github.com/renezander030/skillgate/pull/39","limitations":"One host-agent review using OCR file selection/rules. No OCR-managed model-quality comparison or recall/precision claim."}
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@reneza/skillgate",
"version": "0.11.0",
"version": "0.12.0",
"publishConfig": {
"access": "public"
},
Expand Down
27 changes: 27 additions & 0 deletions schema/done.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,33 @@
},
"gate": {
"oneOf": [
{
"type": "object",
"additionalProperties": false,
"required": ["id", "type"],
"properties": {
"id": { "type": "string" },
"description": { "type": "string" },
"when": { "$ref": "#/definitions/when" },
"type": { "const": "trufflehog" },
"trufflehog": { "type": "string", "minLength": 1 },
"namesFile": { "type": "string", "minLength": 1 },
"timeout": { "type": "integer", "minimum": 1 },
"maxBytes": { "type": "integer", "minimum": 1 }
}
},
{
"type": "object",
"additionalProperties": false,
"required": ["id", "type", "file"],
"properties": {
"id": { "type": "string" },
"description": { "type": "string" },
"when": { "$ref": "#/definitions/when" },
"type": { "const": "review" },
"file": { "type": "string", "minLength": 1 }
}
},
{
"type": "object",
"additionalProperties": false,
Expand Down
41 changes: 37 additions & 4 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { runSync } from "./link.js";
import { runScaffold, listTemplates } from "./scaffold.js";
import { doctor, installIntegration, INTEGRATION_TARGETS, type IntegrationTarget } from "./integrations.js";
import { analyzeCommand } from "./command.js";
import { reviewSnapshot } from "./review.js";
import { readCachedResult, snapshotKey, writeCachedResult, writeReceipt } from "./receipt.js";
import { resolveBaseRef, mergeBase, readFileAtRef, repoRelativePath, repoRoot, isClean } from "./git.js";
import {
Expand Down Expand Up @@ -93,7 +94,8 @@ Usage:
skillgate gate allow/block one command (any harness); exit 2 = block
skillgate gate --event stop block an agent from ending its turn while gates fail
skillgate phase [<id>] show phase status, or move to <id> if its gates pass
skillgate init write an example .skillgate/done.yaml
skillgate init [--preset no-secrets] write an example .skillgate/done.yaml
skillgate review-snapshot print the snapshot to bind an optional review report to
skillgate install <target|all> install agent hooks (claude-code, codex, gemini-cli, cursor,
opencode), github-actions, or pre-commit enforcement
skillgate doctor <target|all> verify policy discovery and one or more integrations
Expand Down Expand Up @@ -432,18 +434,48 @@ if (cmd === "zk-verify") {
}

if (cmd === "init") {
const preset = option("--preset");
if (args.includes("--preset") && preset !== "no-secrets") die(2, "unknown preset; available: no-secrets");
const dir = path.join(cwd, ".skillgate");
fs.mkdirSync(dir, { recursive: true });
const target = path.join(dir, "done.yaml");
if (fs.existsSync(target)) {
console.error(`${path.relative(cwd, target)} already exists`);
process.exit(1);
}
fs.writeFileSync(target, EXAMPLE);
const secretsSpec = `name: no-secrets
finishLine: ["git commit", "git push", "npm publish"]
gates:
- id: no-secrets
type: trufflehog
timeout: 10000
namesFile: .skillgate/forbidden-names.json
`;
if (preset) {
const names = path.join(dir, "forbidden-names.json");
if (!fs.existsSync(names)) fs.writeFileSync(names, "[]\n", { mode: 0o600 });
const ignorePath = path.join(cwd, ".gitignore");
const ignore = fs.existsSync(ignorePath) ? fs.readFileSync(ignorePath, "utf8") : "";
const entries = ["/.skillgate/forbidden-names.json", "/.skillgate/scan-cache/"];
const missing = entries.filter(entry => !ignore.split(/\r?\n/).includes(entry));
if (missing.length) fs.appendFileSync(ignorePath, (ignore && !ignore.endsWith("\n") ? "\n" : "") + missing.join("\n") + "\n");
}
fs.writeFileSync(target, preset ? secretsSpec : EXAMPLE);
console.log(`wrote ${path.relative(cwd, target)} — edit it, then \`skillgate check\``);
if (preset) console.log("Set local forbidden-names.json to a non-empty JSON array of server/domain/tenant names, or remove namesFile for credentials only.");
process.exit(0);
}

if (cmd === "review-snapshot") {
try {
const resolved = resolveSpecAndBase(findSpecPath(cwd));
console.log(reviewSnapshot(resolved.spec, resolved.workspace, resolved.gateBase));
process.exit(0);
} catch {
die(2, "cannot read policy or repository snapshot for review");
}
}

function integrationTargets(value: string | undefined): IntegrationTarget[] {
if (!value) die(2, `integration target required (${INTEGRATION_TARGETS.join(", ")}, or all)`);
if (value === "all") return [...INTEGRATION_TARGETS];
Expand Down Expand Up @@ -574,7 +606,8 @@ if (cmd === "check") {
const timeoutArg = option("--timeout");
const timeoutMs = timeoutArg == null ? undefined : Number(timeoutArg);
if (timeoutArg != null && (!Number.isInteger(timeoutMs) || timeoutMs! < 1)) die(2, "--timeout must be a positive integer in milliseconds");
const cache = args.includes("--cache");
// External state and private denylist files are not covered by the snapshot cache.
const cache = args.includes("--cache") && !r.spec.gates.some(gate => ["trufflehog", "review"].includes(gate.type));
const receipt = option("--receipt");
const receiptFile = receipt ? path.resolve(cwd, receipt) : undefined;
const receiptRel = receiptFile ? path.relative(r.workspace, receiptFile).split(path.sep).join("/") : "";
Expand Down Expand Up @@ -752,7 +785,7 @@ if (cmd === "phase") {
? `no phase gate with id ${wanted}`
: phaseGates.length ? `spec has ${phaseGates.length} phase gates — pass --gate <id>` : "spec has no phase gate");
}
const opts = { baseRef: r.gateBase, gates: new Map(r.spec.gates.map((g) => [g.id, g])), memo: new Map<string, GateResult>() };
const opts = { baseRef: r.gateBase, spec: r.spec, gates: new Map(r.spec.gates.map((g) => [g.id, g])), memo: new Map<string, GateResult>() };
const marker = path.resolve(r.workspace, gate.current ?? DEFAULT_PHASE_FILE);
const current = currentPhase(gate, r.workspace);

Expand Down
9 changes: 8 additions & 1 deletion src/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ import { readFileAtRef, listFilesAtRef, matchesGlob, changedFiles, currentBranch
import { checkManifest, SUPPORTED_MANIFESTS } from "./deps.js";
import { isStructuredCommandMatch } from "./command.js";
import { runShellCommand } from "./process.js";
import { checkSecrets } from "./secrets.js";
import { checkReview } from "./review.js";

export interface GateResult {
id: string;
Expand Down Expand Up @@ -57,6 +59,7 @@ export interface RunOptions {
gates?: Map<string, Gate>;
/** Internal: results already computed in this run, so a required gate runs once. */
memo?: Map<string, GateResult>;
spec?: Spec;
}

export interface PhaseStatus {
Expand Down Expand Up @@ -300,6 +303,10 @@ function checkGate(gate: Gate, cwd: string, opts: RunOptions): GateResult {
}
case "trivy":
return checkTrivyGate(gate, cwd, opts.remainingMs);
case "trufflehog":
return { ...base, ...checkSecrets(gate, cwd, opts.remainingMs) };
case "review":
return { ...base, ...checkReview(gate.file, opts.spec ?? { gates: [gate] }, cwd, opts.baseRef) };
case "evidence": {
const full = path.resolve(cwd, gate.file);
if (!fs.existsSync(full)) return { ...base, ok: false, reason: `evidence missing: ${gate.file}`, location: { file: gate.file } };
Expand Down Expand Up @@ -510,7 +517,7 @@ export function runGates(spec: Spec, cwd: string, opts: RunOptions = {}): RunRes
continue;
}
const gateStarted = Date.now();
const result = runOne(gate, cwd, { ...opts, remainingMs, gates, memo });
const result = runOne(gate, cwd, { ...opts, remainingMs, gates, memo, spec });
results.push({
...result,
status: result.ok ? "pass" : "fail",
Expand Down
4 changes: 2 additions & 2 deletions src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ export const SkillGate = async (ctx: any): Promise<Hooks> => {
try {
spec = loadSpec(specPath);
} catch (e: any) {
// Shell commands fail closed. Other tools stay usable so the policy can be repaired.
if (tool !== "bash") return;
// Keep built-in repair tools usable; unknown/MCP tools may publish and must fail closed.
if (["read", "edit", "write", "apply_patch", "glob", "grep"].includes(tool)) return;
throw new Error(`skillgate blocked tool execution: configured policy is invalid (${e.message})`);
}
// Shell commands cross the finish line by `finishLine`; any other tool by `gatedTools`.
Expand Down
Loading
Loading