feat(pipeline): add repository-declared extra gates - #839
Conversation
Repositories can declare extra checks in .no-mistakes.yaml that run inside
the pipeline, for validation that does not fit an existing step (mutation
budgets, complexity ceilings, architectural fitness functions). Today that
class of check has nowhere to live but commands.test, which the repo-config
reference reserves for targeted validation, or remote CI, where it only ever
runs after push.
A gate is either a command or an agent review, is anchored after a core step,
and can only add a verdict:
gates:
- name: mutation-budget
after: test
command: "make mutation"
- name: arch-fitness
after: lint
instructions: |
No package under internal/ may import internal/cli.
The core sequence is never reordered and never loses a member, so gates make
a pass mean more than the core pipeline, never less. The delivery tail
(push, pr, ci) is unanchorable because a gate running after push would be
validating a branch the world can already see.
A failing gate parks for a decision rather than auto-fixing: the pipeline
cannot know what fixing an arbitrary repository check means, so the finding
stays the author's call.
Gates are honored only from the trusted default-branch copy, regardless of
allow_repo_commands. That opt-in covers a pushed branch re-running its own
suite; a gate instead defines what validating the branch means, and a
contributor must not author the check that clears them.
Refs kunchenguid#805
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; the previous documentation issue has been corrected and no repository-rule violation was identified. Reviews (3): Last reviewed commit: "no-mistakes(document): align pipeline an..." | Re-trigger Greptile |
|
Speaking as Kun's firstmate: Fork CI on HEAD VISION.md per-rule (inspected
Not auto-merging even if CI goes green. This is opt-in extra checks, but it also makes the executor's step list configuration-dependent (pipeline-shape). Agent |
|
Handing this off. The branch is now frozen at What is needed: approve the workflow runs at Why the pipeline could not finish this itselfThe gate's
Combined, the A
|
|
Speaking as Kun's firstmate: Approved workflow runs on frozen HEAD Still not auto-merging even if they go green: pipeline-shape (executor step list is config-dependent) and agent |
2 participants
Closes #805.
Intent
The user wants to contribute a feature to no-mistakes that lets users customize the pipeline's gates: define their own gates and enforce custom steps - custom review steps, custom validation, arbitrary enforcements - on every PR opened through the tool.
Their original request had three parts, and the scope was deliberately narrowed after researching the issue tracker:
ADD custom gates. That is what this change implements, and it is an already-approved request: issue Feature request: an optional per-step command hook, alongside commands.{test,lint,format} #805 asks for an optional per-step command hook for validation that does not fit an existing step (mutation testing, complexity budgets, architectural fitness functions), and its triage labelled it ready-for-pr for "the additive, fail-closed extra-check shape only". Issue Feature request: setup/install command hook for the run worktree #720, the adjacent commands.setup hook, is NOT implemented here.
DISABLE existing core gates. Deliberately NOT implemented and deliberately not requested. The triage on Feature request: an optional per-step command hook, alongside commands.{test,lint,format} #805 states that a hook letting a standing repo config skip or dilute test, lint, format, review, or CI would fail VISION R1 and is out of scope, and issues Feature request: skip or shrink the test-evidence agent pass when a deterministic commands.test exists and passes #529 and Feature: skip document agent when review proves documentation is unaffected #477 were both closed as not-aligned for durable skip configuration. Per-run --skip already covers an explicit one-run skip. Nothing here can switch a core step off; reviewers should expect that omission, it is intentional.
DISPLAY whether a run used default or custom gates, so an owner can still rely on the tool as a safety property across all PRs. Filed separately as issue Feature: declare gate composition in the pipeline attestation so consumers can distinguish a default run from a customized one #838, now also labelled ready-for-pr, and deliberately NOT in this change: it adds a gates object to the pipeline attestation and depends on the config surface this change introduces. Expect it as a follow-up PR.
The user explicitly chose to include agent-driven custom review gates in addition to shell command gates, and accepted that agent gates go beyond the literal shape #805's verdict approved and may draw a scope objection.
Design decisions, all aimed at VISION R1 (a repository may add checks on top but never remove, reorder, or dilute the core):
Verified locally: make lint clean, and go test green on internal/config, internal/types, internal/db, internal/cli, internal/pipeline, internal/pipeline/steps and internal/daemon. Documentation was added to docs/src/content/docs/reference/repo-config.md, which AGENTS.md names as the owner of configuration keys.
This is a single coherent change rather than stacked diffs, because the required PR check binds one pipeline attestation covering review, test and document to a single head SHA.
Status context for this run: this head is already open as PR #839 and has passed the full GitHub check matrix - build, check, e2e, generated-files guard, Greptile 5/5, and tests on ubuntu, macos, windows-core and windows-git. The maintainer's triage independently audited VISION R1 through R7 against the source rather than the PR body and found every rule aligns. The single failing check is "PR must be raised via no-mistakes", which failed only because the attestation in the PR body is bound to an earlier head (cd916e4) than the current one (f0a6805): the prior run's CI step pushed a fix commit and was then cancelled before the pr step could re-render the body. This run exists to re-validate the current head so a fresh attestation is genuinely bound to it. The one open question is a maintainer decision, not a defect: agent instructions gates go beyond issue #805's ready-for-pr scope, which covered a command hook only. They are deliberately retained at the author's direction.
What Changed
gateskey to.no-mistakes.yaml(internal/config/gates.go): each entry names a gate, anchors it to a core step viaafter(rebase,review,test,document,lint— the delivery tail andintentare refused), and supplies exactly one ofcommand(passes on exit 0) orinstructions(an agent judges the change against that rule alone). Names, anchors, duplicate/core-step collisions, the 16-gate cap, and the 16 KiB instructions cap are validated at config parse time on whichever copy is parsed, andEffectiveRepoConfighonorsgatesonly from the trusted default-branch copy regardless ofallow_repo_commands.CustomGateStepandsteps.WithCustomGates, which inserts each gate immediately after its anchor in the run's step list without reordering, replacing, or dropping a core step. A failing gate parks withNeedsApprovalandAutoFixable=false, and every agent-gate finding is forced toask-user; answeringfixruns a fix turn through the sharedexecuteFixModeagainst the previous findings plus the gate's own requirement, then re-runs the check. Gate step names encode their anchor asgate.<anchor>.<label>with a dot separator (a colon would be read as an NTFS alternate data stream in the derived log filename) and share the anchor'sStepName.Order.runs.gates_jsoncolumn pins the resolved gate list at run creation, andloadRecoveredConfigreads it back throughpinnedRunGatesrather than re-resolving from the live default branch;GetStepsByRungained anidtie-break andbuildPipelineAttestationa coreness-first tie-break so a gate sorts after its anchor.axi logs --stepnow accepts a gate name via the newvalidReadableStep, whilevalidStep(and therefore--skip/no-mistakes.skip=) still refuses one, and gate step names collapse to the tokengateinaxi logstelemetry. Docs updated inrepo-config.md(newgatessection),pipeline.md,gate-model.md,pipeline-steps.md,configuration.md,environment.md, andAGENTS.md.Risk Assessment
Testing
Ran the custom-gates e2e journey against a real daemon, real git and a fake agent, which shows the whole feature the way an operator meets it: a maintainer's agent gate and command gate appear as first-class steps immediately after review and test in
axi status, a failing gate parks with every finding forced to ask-user,axi logs --step gate.test.package-registryreturns the gate's own failure output, answering--action fixruns a fix turn and re-runs the gate's check so the branch lands the repair, and a contributor's pushed branch can neither delete the maintainer's gates nor author its own. A gate anchored after push fails its own run naming the five valid anchors. I added targeted manual verification for the two surfaces the unit tests could not show as product output: the published attestation for a gated run was rendered by the real PR-body renderer and accepted by the realrequire-no-mistakes/verify.pywith each gate listed after its anchor (and still correctly rejected when a core step is incomplete), and thegates:example in repo-config.md was resolved through the real parser and step-list builder to confirm it produces exactly the step name the docs tell operators to type. Alongside these, the gate-related unit tests across types, config, db, cli, pipeline/steps and daemon are green, including gate-pin recovery under a changed default branch. The only change without visual evidence is the repo-config.md prose, because rendering the Starlight docs site requires a full network npm install into the worktree that would be disproportionate for a reference-page section; its claims were verified against the code instead.Evidence: Operator transcript: gated run from axi status through the authorized gate fix to the pushed branch
Evidence: Full e2e journey log (TestCustomGatesJourney, 3 subtests, all pass)
Evidence: Generated PR body section for a gated run — each gate sits immediately after its anchor in the v1 attestation
no-mistakes-pipeline-attestation:v1 {"head_sha":"762b8638...","steps":[{"step":"intent","status":"skipped"},{"step":"rebase","status":"completed"},{"step":"review","status":"completed"},{"step":"gate.review.arch-fitness","status":"completed"},{"step":"test","status":"completed"},{"step":"gate.test.package-registry","status":"completed"},{"step":"document","status":"completed"},{"step":"lint","status":"completed"},{"step":"push","status":"completed"},{"step":"pr","status":"completed"},{"step":"ci","status":"completed"}]}