Untrack `pkg/rt/generated.sums` once #683 merges

[mparrett]

Untrack pkg/rt/generated.sums once #683 merges

• Depends on: build: content-gate every committed generated artifact, not just three #683 (content-gate every committed generated artifact). Supersedes the original dependency on build: content-gate every committed generated artifact, not just the bundle #634, closed 2026-08-05.
• Epic: Epic: Maintenance debt — retire superseded mechanisms, reduce process weight #528

Summary

pkg/rt/generated.sums is a committed digest of every .lg and generator source. #217 introduced it to replace mtime-based freshness checks, which are unreliable after a git or jj checkout, and it solved that problem.

This proposes retiring the committed copy: gitignore the file, have make generate write it locally, and remove the merge driver that reconciles it. The digest stays. It stops being shared state in the repository.

What the committed copy costs

The digest records a claim about history, and a rebase invalidates that claim by construction. The recorded value means "the artifacts were generated when the sources hashed to X". A rebase rewrites history while leaving artifacts and digest untouched, so the claim goes false with nothing in the tree wrong. The PR goes red on a healthy checkout.

The merge=sums driver only fires when git reports a conflict on the file. When a rebase drops or collapses the digest commit instead, no conflict occurs, the driver never runs, and the digest goes stale silently. That is why the failure reads as intermittent, and why "run make install-hooks" does not help the people hitting it.

The digest hashes sources whose artifact is not tracked. pkg/rt/core_go_lowered/ is gitignored. Editing pkg/rt/core/ir/passes/dce.lg turns CI red while changing no committed artifact. The IR workstream sits in that path, which is where the friction concentrates.

Three CI gates enforce it today, so partial measures do not help: the build job's make check-generated-manifest step, TestGeneratedArtifactsAreFresh in the test lanes, and make check-generated.
#683 removes the third. The remaining two read the committed file directly, and
only untracking clears both.

Measured. #614 and #625 went red on inherited digests with healthy trees.
#630 and #634 each needed a manual regenerate-and-push after their base moved,
#634 twice.

Why now rather than at #217

The original guardrail matched the project it was written for. With two contributors and same-day merges, a hash-everything digest costs nothing: regenerate, push, move on. With multiple contributors, forty-plus merges a week, and PRs open across days of upstream movement, the same file becomes shared mutable state that every compiler-adjacent PR collides on. The guardrail did not get worse; the traffic through it changed.

Ordering: #683 first

The digest is currently the only signal covering op_generated.go, ir_bridge_generated.go, zz_primitives_generated.go and core/ir/data/generated.lg. Untracking it now would remove real protection.

With #683 in, make check-generated compares all five committed artifacts byte for byte — a direct check rather than a proxy.

Proposed change

• Gitignore pkg/rt/generated.sums and remove it from the index.
• Keep make generate writing it and scripts/pre-commit reading it as a local fast probe, where answering in milliseconds is the point.
• Remove .gitattributes merge=sums, scripts/git-merge-sums.sh, and the merge.sums.* half of make install-hooks. Existing clones keep a dead merge.sums.* entry until cleared by hand — harmless, worth noting in the PR.
• Drop the build job's manifest step and TestGeneratedArtifactsAreFresh, both of which read the committed file. See the open question for what replaces the local signal.

Open question: the local baseline

Untracking removes the baseline that two local checks read. scripts/pre-commit has nothing to compare against on a fresh clone, and TestGeneratedArtifactsAreFresh fails outright with no manifest present. That test is #217's real safety net, and the reason staleness surfaces in a plain go test ./... instead of a 67-second gate.

So this has a cost at the inner loop: after the change, editing a .lg without regenerating gives no local signal until make check-generated or CI. Both checks need the same baseline, so I would settle it once. Three options, no strong preference:

1. Regenerate once on first run and cache locally.
2. Fall back to the full content gate when no local digest exists, paying that cost on the first commit only.
3. No-op until a local digest exists, making it opt-in.

Option 1 keeps the fast path but puts a full regeneration in front of someone's first commit. Option 3 is least surprising and weakens both checks for anyone who never runs make generate. Worth settling before the PR.

Not in scope

core_compiled.lgb stays tracked and still conflicts on rebase — it is //go:embed-ed and go install needs it in the tree. The lgb merge driver stays. This covers the digest only.

[mparrett]

The committed digest serializes merges, and today put a number on what that costs.

Before this morning, six open PRs edited pkg/rt/generated.sums: #628, #641, #649, #675, #711, and #712. I simulated every pair against main, and every pair conflicts. The body makes the staleness case; this is the stronger one. Of those six, exactly one could merge without a rebase, and the other five had to wait for it and pull.

I merged #628 (f3ca5f9b). Four of the remaining five went from mergeable to conflicting on generated.sums: #649, #675, #711, and #712. Each was green, and each needed one rebase. #675 has since been rebased; the other three still conflict as of a9d0b230.

Two of the four were approved and merge-ready when they were knocked back. #711 was approved at 04:58Z and #712 at 14:25Z; #628 merged at 14:50Z. Neither had anything wrong with it. Merging one PR cost two finished ones belonging to someone else, and nothing in the merge flow surfaces that cost at the point of deciding.

So the cost is throughput, not only wasted CI: the file makes compiler-adjacent PRs mutually exclusive, and review capacity stops being what sets the merge rate.

The driver hides conflicts GitHub will still hit. The body notes that merge=sums only fires when git reports a conflict. When it does fire, git merge recomputes the digest and reports success while GitHub, which has no such driver, reports the PR as conflicting. I hit this on #624, #641, and #649, assumed GitHub was serving a stale mergeability cache, and settled it by re-running the merges with -c merge.sums.driver=false — which reproduces GitHub's failure exactly. "It merges clean locally" says nothing about the merge button, and anyone debugging a red PR on their own machine gets told everything is fine.

Ordering. This depends on #634, which is closed — superseded by #683 on 2026-08-05, which carries the same content gate and is open. The dependency holds; it points at #683 now.

On the open question about the local baseline: option 2, fall back to the full content gate when no local digest exists. Option 1 puts a full regeneration in front of someone's first commit, which is the worst moment to spend it, and option 3 goes quiet for exactly the contributors least likely to notice. Option 2 pays once, visibly.