build: content-gate every committed generated artifact, not just the bundle

[mparrett]

make check-generated verified one of five committed generated artifacts. It regenerates through lgbgen, which emits only core_compiled.lgb and the gitignored lowered tree, so op_generated.go, ir_bridge_generated.go, zz_primitives_generated.go and core/ir/data/generated.lg were never compared against their sources.

One had drifted. cmd/lginterop/prims_emit.go emits an ungrouped import block, and its comment assumed "the final file is run through gofmt, which canonicalizes the import block". gofmt preserves existing grouping rather than introducing it; only goimports inserts the stdlib separator, and goimports is a linter here, not part of generation. The committed copy was hand-grouped, so every make generate reproduced a file the linter would rewrite — anyone regenerating and committing would have broken lint.

Change

• Emit the blank line in the generator, making regeneration idempotent.
• GENERATED-TRACKED lists the five committed artifacts. The gate stashes them, regenerates through make generate, and cmps each. Using the full pipeline rather than lgbgen alone is what widens the coverage.
• check-generated-manifest is no longer a prerequisite of check-generated.

On the prerequisite

This does not change whether a stale digest fails CI. It still does, in the build job (which runs check-generated-manifest directly) and in TestGeneratedArtifactsAreFresh. Neither is touched here.

What changes is that a stale digest no longer aborts check-generated before the content gate can run. The digest is a proxy; the cmp is the check that binds sources to artifacts. The proxy should not suppress the answer.

pkg/rt/generated.sums is stashed and restored rather than compared, because lgbgen's writeBundle refreshes it on every regeneration — without that, a check-* target would mutate a tracked file.

Verification

• Passes clean; all five report lockstep and the tree is unmodified afterwards.
• Catches injected drift in op_generated.go and zz_primitives_generated.go, neither of which the previous gate could see.
• A failing generator now prints the underlying compiler error. It previously printed only make[1]: *** [generate] Error 1.
• All five artifacts are byte-deterministic across consecutive make generate runs.
• Cost is flat: regeneration goes 55.4s to 56.6s, whole gate 67s, against a job with no timeout set.

Note on the recipe

It runs under plain /bin/sh with no -e. The makeplus branch that sets SHELL := bash and -e -o pipefail is skipped whenever go is already on PATH, which is every CI run. mktemp and cp failures are checked explicitly and a trap removes the temp dir on interrupt.

[mparrett]

Follow-up filed as #635: untrack the digest once this lands. This PR is the prerequisite — it moves the four other committed generated artifacts onto a direct content check, so the digest stops being the only thing guarding them.

[mparrett]

Reworked after self-review. Two regressions fixed: the sub-make's >/dev/null was swallowing generator diagnostics, and the gate was rewriting pkg/rt/generated.sums as a side effect of make generate.

Also corrected the framing. I had claimed dropping the check-generated-manifest prerequisite would stop rebased PRs going red. That was wrong — a stale digest still fails the build job (which runs the target directly) and TestGeneratedArtifactsAreFresh. This PR only changes whether the proxy aborts the content gate. Removing the rebase friction is #635's job, not this one.

[mparrett]

Filed #637 for the follow-up: GENERATED-TRACKED is hand-maintained and nothing checks it against scripts/generate.lg, so a sixth committed output would silently go uncovered. Kept out of this PR to avoid another review round.

[mparrett]

Converting to draft; can re-open after current in-flight backlog lands.

[mparrett]

Coordination status

nnunley’s active stack is #639 → #640 → #641, with #627 folded into #639. The generated-output gate overlap begins in #640 and is extended by #641’s dependency-manifest work.

I proposed two possible integration orders in the #641 discussion, but a direction has not selected one yet. I’m putting this PR back into draft and holding the integration decision until we confirm the branch owning gate reconciliation, so this PR does not force churn into the active stack.

[nooga]

Requesting changes — this branch has gone from "waiting on #639/#640/#641 to settle" to structurally obsolete now that #639 and #640 have merged.

• cmd/lginterop/prims_emit.go, the file this PR patches for the import-grouping bug, no longer exists on main — it was moved/refactored to internal/primgen/prims_emit.go under a new cmd/lgprimgen tool by #639/#640.
• The specific bug this PR fixes (missing blank line between stdlib/internal imports) has already been independently fixed at the new location.
• The underlying goal — generalizing check-generated past just the bundle — is still valid and still missing on main. In fact main's check-generated has grown a third copy-pasted stanza (pkg/rt/corefns/zz_primitives_generated.go, from #640) on top of the original bundle-only gate, which is exactly the anti-pattern this PR was meant to replace.

Given how much of the target code has moved, I think a rebase would end up rewriting most of the diff anyway — recommend closing this and opening a fresh PR against current main that folds in the corefns registrar and the renamed lgprimgen invocations. The value proposition here is still real, just the diff isn't.

Two smaller things worth carrying into the redo, either way:

• Makefile's check-generated recipe doesn't restore originally-committed bytes on a detected STALE failure — leaves a dirty multi-file tree with no printed guidance to git checkout it.
• docs/contribution-policy.md:89-90 still describes the gate as bundle/lowered-tree only.

[mparrett]

Closing in favor of #683. @nooga's read was right: cmd/lginterop/prims_emit.go moved to internal/primgen/ in #639/#640, the import-grouping bug it patched has already been fixed there, and a rebase would have rewritten most of the diff.

#683 carries the thesis against current main — one GENERATED-TRACKED list replacing the now-three copy-pasted stanzas, regenerating through the full make generate so the three artifacts only generate.lg produces stop being invisible. Both carry-overs are in it: the gate restores the working tree on a failure instead of leaving it dirty, and docs/contribution-policy.md is updated.

Widening it turned up a second defect worth naming here. make generate did not regenerate the corefns registrar from #640 — it was reachable only through the Makefile's core_compiled.lgb prerequisite, an mtime rule that stops firing once the committed registrar is newer than its sources. So check-generated could report the registrar stale while the fix it printed, "run make generate", could not repair it. That had to be fixed in #683 regardless, because a gate regenerating through make generate would otherwise have passed corefns drift silently.