feat(ir): def+name* IR-compile seam + block-junk agreement (salvaged from #556) by nnunley · Pull Request #625 · nooga/let-go

nnunley · 2026-07-23T03:28:33Z

Two unmerged IR/compiler changes that a whole-working-copy snapshot had swept into #556's "self-heal submodule" commit. #556 is now cleaned to just the submodule work; these were extracted here so they aren't lost. Neither exists anywhere else in the tree.

They're independent and could be split into two PRs — happy to if preferred. Flagging up front so review can weigh that.

1. def+name* IR-compile seam

compiler.go maybeIRCompileDefFnArg + pipeline.lg compile-def-fn-value: when *ir-compile* is on and a top-level (def NAME (name* … (fn …) …)) is compiled, route the inner fn through the Lisp IR pipeline and substitute the compiled Fn as an embedded constant — the same def-value shape defn produces via chunk->fn.

The defn macro can't cover these: def is a special form, so grammar-style rule defs (the yamlstar/grammar def-closures) never reach the macro layer and otherwise always compile via the plain bytecode compiler. Any failure falls back silently, mirroring defn's hybrid path. Top-level defs only — an inner fn compiled through the pipeline can't capture enclosing locals.

2. block-junk agreement (`lower.lg`)

record-block-junk! now requires every fall-through predecessor to agree on junk-below (a disagreement aborts this fn's lowering, dropping to the plain bytecode compiler) instead of silently taking the max. Junk is baked into the target's OP_RECUR drop counts, so a path entering with different junk over- or under-drops the runtime stack — observed as slice bounds out of range underflows. Take-the-max was only safe while mismatching shapes couldn't lower at all; the deferral/RPO fixes let them lower, so the disagreement now has to be caught. RECUR-edge preds still normalize to junk 0, as before.

Review note: this touches the same lower.lg stack-discipline area as #579 (now merged). The branch is 3-way-merged with #579 — both load-var-crosses-effect? and the :block-junk-seen agreement are present — but the interaction wants a careful look.

Testing

Rebased on current main; make generate + make check-generated clean; both build tags compile; go test ./pkg/ir ./pkg/vm ./test green.

Branch name reflects its salvage origin (#556); rename on request.

…ted from nooga#556 mis-snapshot) Two unmerged IR/compiler features that a bad whole-working-copy snapshot swept into nooga#556's 'self-heal submodule' commit (pxvytswo). Neither is in main; neither belongs in a submodule PR. Extracted here to preserve them for a proper PR home + review — NOT ready as-is: - compiler.go maybeIRCompileDefFnArg + pipeline.lg compile-def-fn-value: route top-level (def NAME (name* ... (fn ...))) grammar defs through the IR pipeline (the def+name* / grammar-def-closure AOT seam). - lower.lg :block-junk-seen: fall-through predecessors must AGREE on junk-below (disagreement aborts lowering) instead of take-the-max — fixes a stack underflow. Overlaps nooga#579's lower.lg stack-discipline rework; needs re-review against it. TODO: decide home (grammar/yamlstar effort?), split into 2 PRs, review vs nooga#579.

mparrett

Really glad to see these salvaged rather than lost. A whole-working-copy snapshot is exactly how good work gets orphaned, and rescuing it is the right call. The individual pieces are thoughtful, and the comments walking through the stack-discipline reasoning are genuinely helpful to review against.

My main ask is re: packaging. As it stands the diff does more than the title implies. I count six logically independent changes, and four aren't called out in the description:

def+name* IR-compile seam (compiler.go + compile-def-fn-value) — described, and cleanly guarded: top-level-only, single-fn, silent fallback, FormSource preserved. This one reads as self-contained.
block-junk agreement in record-block-junk! (throw-on-mismatch → fallback) — described.
RPO block emission order (rpo-block-order + :next-of), replacing raw id-order emission.
deferrable-branch-if-cond? new guards (block-arg exclusion, branch-arg-of-own-terminator exclusion).
tail-call fusion (fused-tail-call + chunk-emit-pop-n) — a new optimization: :return of a single-use :call → TAIL_CALL.
*runtime-defn-ir-cache* single-ns retention (pipeline.lg) — a heap/GC tweak.

3–6 all land in the lower.lg stack-discipline core, and 3/4/5 each fix or add miscompile-class behavior (the underflow, the and-cond-loop RPO case, the block-arg cond that BRANCH_F would mis-pop, tail-call parity). Bundled into one salvage commit with no shape-specific tests, that's a lot of surface to land at once in the area we're most careful about. Green CI is necessary but not sufficient here — a coverage census won't catch tail-call fusion or RPO computing a wrong result.

Two requests; happy to help with either:

Split it. You offered — I'd take you up on it and go a bit further than the 2-way: (a) the compiler seam [1], (b) the lower.lg correctness bundle [2–4], (c) tail-call fusion [5], (d) the inline-cache retention tweak [6]. Each can then merge on its own evidence. If a full split is too much churn, even peeling the tail-call-fusion optimization [5] and the cache tweak [6] out from the correctness fixes [2–4] would help a lot.
Tests. A shape-specific regression per fix — the junk-mismatch underflow repro, the and-cond loop that RPO fixes, the block-arg cond deferral case, and a tail-call-fusion parity test vs the plain compiler — would pin exactly what each change is for and keep it from silently regressing.

One coordination note: changes 2–4 move how much lowers to bytecode, which is the same path #580's new census baselines. Whichever of #580 / #625 lands second will want to rebaseline ir-stress-bytecode-baseline.edn — worth a heads-up so the ratchet shift is expected rather than a surprise.

None of this is a knock on the work. It's solid. I think it's easier to land in pieces we can each verify. Let me know hwat you think.

nnunley · 2026-07-30T15:26:22Z

Split into 4 independently-reviewable PRs per your packaging request, @mparrett:

[1/4] def+name* IR-compile seam — feat(ir): def+name* IR-compile seam — top-level (def x (fn …)) via the IR path #647
[2/4] lower.lg correctness bundle (block-junk agreement + RPO order + deferrable-branch guards) — fix(ir/lower): block-junk agreement + RPO block order + deferrable-branch guards #648
[3/4] tail-call fusion — perf(ir/lower): tail-call fusion — :return of a single-use :call → TAIL_CALL #649 (stacks on [2/4]; both touch lower.lg, so perf(ir/lower): tail-call fusion — :return of a single-use :call → TAIL_CALL #649's diff includes fix(ir/lower): block-junk agreement + RPO block order + deferrable-branch guards #648 — review [2/4] first)
[4/4] *runtime-defn-ir-cache* single-ns window — perf(ir): retain *runtime-defn-ir-cache* for only the current ns #650

[1/4], [2/4], [4/4] are independent off main with clean single-change diffs; [3/4] stacks on [2/4]. Each carries its own regenerated artifacts and builds under both VM variants.

Tests: [2/4] adds test/ir_blockarg_cond.lg — red-checked, it fails on main with the exact index out of range [-1] stack underflow the deferrable-branch guard fixes. [3/4] adds test/ir_tailcall_fusion.lg, a parity guard vs the plain compiler.

On the other two fixes' tests: I couldn't reproduce the block-junk-agreement or the RPO miscompile from hand-written Lisp — they read as guards for mis-shapes that only arise from complex generated IR, so the ir-stress census is their real guard rather than a .lg unit test. If you have a source-level shape that triggers either, I'll add it. Details in #648, along with the #580 ir-stress-bytecode-baseline.edn rebaseline heads-up.

Closing in favor of the four.

….)) via the IR path Extracted from nooga#625 [1/4]. Adds the compiler.go seam that routes a top-level single-fn def through *ir-compile* (top-level-only, single-fn, silent fallback, FormSource preserved) plus the ir_bridge.lg hook.

…anch guards Extracted from nooga#625 [2/4] — the lower.lg stack-discipline correctness bundle: - record-block-junk! now requires agreement (throw-on-mismatch → fallback) instead of taking the max; RECUR back-edges seed junk=0. - Block emission walks reverse-postorder (rpo-block-order + :next-of) so junk counts are recorded in dependency order — fixes the and-cond-loop miscompile. - deferrable-branch-if-cond? excludes block-arg conds and branch-args of the block's own terminator (a BRANCH_F would mis-pop them).

Extracted from nooga#625 [4/4]. Drops the process-lifetime retention of the runtime defn IR cache to a single-namespace window — a heap/GC tweak, no behavioral change.

…anch guards Extracted from nooga#625 [2/4] — the lower.lg stack-discipline correctness bundle: - record-block-junk! now requires agreement (throw-on-mismatch → fallback) instead of taking the max; RECUR back-edges seed junk=0. - Block emission walks reverse-postorder (rpo-block-order + :next-of) so junk counts are recorded in dependency order — fixes the and-cond-loop miscompile. - deferrable-branch-if-cond? excludes block-arg conds and branch-args of the block's own terminator (a BRANCH_F would mis-pop them).

Disposition for nooga#648 rebase: - Remove 'nooga#625 fix N' references from test headers - Clarify strict-mode claim in ir_junk_agreement test - Regenerate artifacts Exit code: 0

…IL_CALL Extracted from nooga#625 [3/4]. In lower-block!, when a block's :return wraps a single-use :call that is the block's last live instruction, emit TAIL_CALL directly (RETURN kept for shape parity) instead of CALL+RETURN. Stacks on the RPO/block-junk correctness fixes [2/4]; parity-checked against the plain bytecode compiler.

….)) via the IR path Extracted from nooga#625 [1/4]. Adds the compiler.go seam that routes a top-level single-fn def through *ir-compile* (top-level-only, single-fn, silent fallback, FormSource preserved) plus the ir_bridge.lg hook.

Extracted from nooga#625 [4/4]. Drops the process-lifetime retention of the runtime defn IR cache to a single-namespace window — a heap/GC tweak, no behavioral change.

Disposition for nooga#648 rebase: - Remove 'nooga#625 fix N' references from test headers - Clarify strict-mode claim in ir_junk_agreement test - Regenerate artifacts Exit code: 0 fix(ir/lower): rebaseline ir-stress-bytecode after block-junk agreement fix Updated docs/perf/ir-stress-bytecode-baseline.edn to 2026-08-04 results: - Total: 2467 fixtures (+1 new test fixtures) - Failures: 720 (+1 from baseline 719) The block-arg bucket remains at 20 (not the +79 movement in mparrett's initial analysis — that was against an intermediate state; current branch is stable). Minor variations in other buckets (call 428→430, eq 68→69, try 145→144) are typical baseline drift. The deferrable-branch guard (block-junk agreement fix) correctly requires block-arg stack positions for merge blocks with mismatched fall-through stack effects, trading strict lowering requirements for runtime correctness.