Epic: Maintenance debt — retire superseded mechanisms, reduce process weight

[mparrett]

Umbrella for behavior-preserving cleanup: mechanisms that have been superseded but never retired, logic that exists in several copies, and build/CI weight that outlived its reason. Distinct from runtime performance and concurrency (#464), lowering/codegen (#258), and Clojure-semantics parity (#257): nothing here changes what programs do — only how much code and process we maintain to do it.

The recurring pattern, across otherwise-unrelated corners: a migration lands and the predecessor stays. Each leftover is small; together they're where a lot of the "why are there two of these?" questions come from. A roll-up makes them visible and cheap to veto before anyone spends PR effort.

The lists below are the instances I've verified so far — exemplars of the pattern plus known finds, not an exhaustive inventory. New sightings accumulate here as sub-issues.

Gate for everything under this epic: behavior unchanged. Anything that would redraw a behavior line (however small) gets its own issue and discussion first.

Superseded but still alive

• Per-file //go:embed source vars (AsyncSrc, IRBuildSrc, IROpsSrc, the 19 IR pass vars, and friends) superseded by the all:core FS embed in core_embed_fs.go. Roughly 33 vars with zero references (including the whole files irpasses.go, irbuild.go, irops.go, asyncembed.go), and each embeds a second copy of its core source into every non-bootstrap binary. Only CoreSrc is live.
• scripts/lginterop.lg, superseded by cmd/lginterop Corrected (see comments): it's the live codegen back end — cmd/lginterop reads it from the repo root on every invocation (writeGenScript, main.go:419). The item is now code homing: //go:embed the macro library (or move it under cmd/lginterop/) so the binary is self-contained and the pairing is visible from both sides.
• The wasm/ playground: pre-LetGoHost design (xterm 5.3, bare window.Eval, no COI/worker path) maintained in parallel with the pkg/rt/wasm assets that lg -w ships. pages.yml deploys the old one as the homepage. Proposal: regenerate the playground from the current assets and delete wasm/main.go + wasm/index.html. (This one touches the public playground, so it's a "needs a call" item: the page would gain the COI/worker boot path.)
• Two pre-commit systems: scripts/pre-commit (symlink install, runs check-generated) and .pre-commit-config.yaml (prek). README and the YAML give contradictory install instructions; check-generated is already enforced by the unit test and CI. Fold the hook into the YAML as a local hook and keep one documented install path.
• pkg/rt/core/ir/ops.lg ("S3 walking skeleton"): the op-registry dispatch it introduces still coexists with the cond arms it was meant to replace. Finish the migration or delete the skeleton. (On hold: a larger ops-system refactor is queued for after the 1.12 cut — see comments.)

Dead code

• pkg/ir/spike_rpnvm_test.go + bench (~4k lines): the RPN-VM research spike whose conclusions now live in the gogen path.
• check-lowered-fresh Makefile target: the Makefile's own comments describe it as a silent no-op pointing at a path that no longer exists.
• scripts/turnaround-ratchet.lg: no Makefile/CI/hook reference, and its baseline file (docs/perf/turnaround-baseline.edn) doesn't exist. Its openspec change dir is likewise stale — as is openspec/changes/purify-clojure-core/, which shipped in refactor(rt,core): purify clojure.core — let-go.core / let-go.types for lg-isms #410 and can be archived.
• tap_migrations.json at the repo root: a Homebrew tap-migration map with zero references in-tree; belongs in the tap repo.
• Zero-caller odds and ends: compiler.NewDebugCompiler (and the permanently-false debug branch it feeds), vm.MakeFloat, and the if smart branch in cmd/lginterop that emits byte-identical output on both sides.

Same logic, several copies

None of these are style-level DRY: in each case the copies encode one invariant that must change in lockstep (or two binaries that must behave identically), and the collapse adds no new package dependency: every consumer already imports the package the helper would live in. Where duplication is deliberate it stays (the frozen v2Stride migration table is frozen precisely so it does not track the live enum).

• Opcode stride tables ×4: CodeChunk.Debug (vm.go), rt/disasm.go, wasmhost/request.go (byte-identical to the disasm copy), plus the deliberately frozen v2Stride in the migration code. Instruction widths are a single source of truth; a drifted copy silently misparses bytecode. Export one vm.OpcodeStride and keep only the frozen table separate.
• The "infer bundle signature, maybe remap" sequence appears in four decode paths in pkg/bytecode/decoder.go; the next opcode migration has to be threaded through all four. Same-file extraction, one helper.
• The bundled-self-exec boot in lg.go near-duplicates cmd/lg-runtime's runBundle (same store-id fallback, resource decode, args plumbing). lg-runtime's contract is "identical to lg minus the compiler," so the resolver install is the only intended difference. Extract a shared helper parameterized on exactly that, making any other divergence impossible rather than latent.

Process weight

• The Makefile clones makeplus/makes unpinned from GitHub at build time and includes it. go.yml's own comments document removing exactly this pattern for the Clojure toolchain; the Go build path still has it. Pin to a SHA or vendor the few .mk files used.
• Generation freshness is checked and then regenerated anyway in the same CI build job, and check-generated-manifest duplicates the front half of check-generated. One check, one regeneration.
• The Go version is pinned in three places (go.mod, Makefile GO-VERSION, mise.toml).

Related, tracked elsewhere

• Desugar try/catch (and other frontend-duplicated sugar) at the Clojure level — try as a core macro over a try* kernel #477 — single-sourcing the frontend-duplicated sugar (try/catch, destructuring, defn grammar). The biggest same-logic-several-copies item; it has its own design, so it stays there.
• Annotation-driven primitive registrar + Fn-adapter (finish namespace cleanup; improve lginterop model) #411 — annotation-driven registrar and the namespace-cleanup tail.
• Epic: Runtime performance & concurrency #464 — runtime perf and concurrency, including the race-detector CI gap (ci: run the race detector over pkg/vm and pkg/rt #524).
• Write more of let-go in let-go: capability unlocks for .lg + AOT lowering #519 — moving more of the runtime into .lg; several items above shrink the Go surface that work has to cover.

Workflow

Same as #464: file maintenance-debt issues as sub-issues here, each standing on its own (what's dead/duplicated, the evidence, the proposed removal or collapse). I'll take the first batch: the dead-code group is grep-verifiable and PR-sized. The proposals above are my default shapes, open to different cuts. If any item is intentional rather than leftover (the wasm/ playground's separate identity, the openspec archiving convention, the migration-registry posture), a one-line veto here saves a PR round-trip.

[nnunley]

I would be careful with some of the superceded notions, specifically since lginterop generates go code.

- cmd/lginterop (Go binary) is the front end. It reads deps.edn/-packages, uses go/types to scan the target Go package, and extracts the exports into compact positional vectors ([:func "Name" [type-params] [params] [results] variadic?], etc.).
- scripts/lginterop.lg is the code-generation back end. main.go's writeGenScript (cmd/lginterop/main.go:419) reads scripts/lginterop.lg verbatim, appends (def exports [...]) plus a call to (lginterop/generate ...) into a temp .lg file, and shells out to the lg binary to run it. The .lg side builds a gogen AST and spits the final Go file.

The thing that is generated is the output of both together: the interop_*.go files like pkg/rt/interop_glplat.go ("Code generated by lginterop -packages ..."). So the flow is:

cmd/lginterop (scan via go/types) → exports EDN → lg + scripts/lginterop.lg (gogen emit) → pkg/rt/interop_<pkg>.go

The likely source of the confusion: the checkout in the default workspace currently doesn't have scripts/lginterop.lg on disk (whatever branch the other agent has it on predates or drops it), while cmd/lginterop/ is present — which makes it look like the script was replaced by the cmd. On main@upstream both exist, and deleting scripts/lginterop.lg would actually break the binary at runtime, since it reads that file from the repo root every invocation rather than embedding it.

I think it's a good pointer for code homing consolidation

[nnunley]

Additionally: I have a large refactoring of the ops system that I've been holdig off until we make the 1.12 cut that cuts down a lot of the IR duplication. That said, there's still a lot of dangling code.

[mparrett]

Verified, and you're right — the body had it wrong. writeGenScript (cmd/lginterop/main.go:419) reads scripts/lginterop.lg from the repo root on every invocation, so it's the live codegen back end, not a superseded predecessor. My check covered Makefile/CI/generate-script references and missed the runtime file read; #467's corpus exclusion made it look retired. I've edited the body: that bullet is now the code-homing item you pointed at — the generator's two halves live in cmd/lginterop/ and scripts/, coupled by a repo-root file read, so //go:embed-ing the macro library (or moving it under cmd/lginterop/) would make the binary self-contained and remove the exact trap that caught me.

Good to know about the ops refactor. I'll keep the ops.lg and IR-duplication items out of my batches until the 1.12 cut lands, and start with the dead-code group, minus this one.

[nnunley]

One correction to the "Superseded but still alive" list, plus a proposal filed as a sub-issue:

scripts/lginterop.lg is not superseded by cmd/lginterop — it's a live runtime dependency of it. The external-package mode reads the file from the repo root on every run (writeGenScript, cmd/lginterop/main.go:419), concatenates the scanned exports onto it, and shells out to a built lg binary to render the interop Go file (e.g. pkg/rt/interop_xxh3.go). Deleting it today breaks lginterop -packages. The #467 corpus exclusion was about ir-stress coverage, not liveness.

The retirement of the awkward parts (repo-root file read, temp-file concat, shell-out to a pre-built lg) is still right — but as a relocation, not a rewrite: the emitter stays in lg (embed it in cmd/lginterop, evaluate in-process; end state is AOT-lowering it to pure Go via the existing core_go_lowered pipeline). That, plus a plan for draining the ~384 imperative ns.Def sites in lang.go into the #411 registrar, is written up in #531.

[nnunley]

Correction to my correction, after actually running it: lginterop -packages is already broken on main — scripts/lginterop.lg fails to compile against the current form-based gogen API (gogen/parse-stmts was removed), so interop_xxh3.go can't be regenerated at all today. Filed as #535 with root cause and fix direction (repair the .lg against the new API — prerequisite for #531 S1's byte-identical golden test). So the epic's instinct was righter than my first comment allowed: the file is live wiring but dead in practice until repaired.

[nnunley]

Aligning a few items with the IR roadmap on my side, since three of them intersect it — two claims/vetoes and one soft hold, everything else green from me.

pkg/rt/core/ir/ops.lg (S3 walking skeleton) — claim, veto deletion

This is the "larger ops-system refactor queued after the 1.12 cut" mentioned in the item, and it's real and in flight on my side (local stack, ~9 commits): local-carrying? as a catalog column, facet columns + reified infer facets + a :try lower re-home, and a catalog-driven OpByKeyword switch. The skeleton/cond-arm coexistence is mid-migration, not abandonment — deleting the skeleton would orphan the finished half. Finish is the resolution, and the finishing work exists.

Sequencing: it rebases after #558 lands (the reified infer facets sit on the type-of/set-type! accessor seam that PR keeps stable, so the rebase is clean — reviewed there), and after the 1.12 cut as already noted.

North star, for context on several ir-adjacent items at once: the op catalog (defop/ir_ops.lg) becomes the single source of truth for every per-op facet — build, infer, lower, props — with exhaustiveness enforced at generation time instead of silent cond fall-throughs. Beyond that, ops (and eventually IR insts themselves) become native types via the deftype→Go-struct lowering path (working skeleton exists: capture→interface/struct/ctor, devirtualized dispatch, bytecode parity green). #558's side table is the interim storage step on the same arc: tuple + side tables today, packed native structs later.

Two consequences for this epic:

• pkg/rt/core/ir/data/generated.lg (the keyword-map inst layer, flagged in perf(ir): move inferred types to a positional side table — ~100x faster type writes #558 / ir.data.generated has drifted structurally from the active IR data layer — retire or rehabilitate #566): retire, don't rehabilitate — the deftype conversion replaces it at a different level, so rehabilitating the generator would be building a second bridge to a shore we're leaving.
• The opcode stride tables ×4 collapse to vm.OpcodeStride: good now, and compatible — the catalog eventually generates that one table (bytecode facts are op facets), so collapsing first means the generator later targets a single site.

pkg/ir/spike_rpnvm_test.go — soft hold, or delete-with-pin

The index-RPN execution-form question is queued for a re-check, and recent findings strengthen the case for actually running it: the stack lowering's junk-below accounting has proven fragile (per-pred disagreement corrupts RECUR drop counts), and IR-lowered bytecode measures ~2× the instruction count of the direct compiler on branch-heavy code, dominated by branch-arg DUP shuffle. An IR-direct/register-style execution form dodges that class by construction — and the spike is the harness we'd re-run to decide. Either hold until that verdict, or delete now with the SHA pinned in this issue so resurrection is one revert (conclusions are recorded; the 4k lines don't need to sit in-tree meanwhile — pin-and-delete is fine by me if you note the SHA).

Everything else

Embed-var dedup, decoder helper extraction, lg/lg-runtime boot helper, pre-commit unification, process-weight items: no interaction with the IR roadmap — go for it. On the pre-commit item specifically, one more data point for the fold-up: the prek pre-push go test -timeout 60s ./... can no longer fit the e2e suite (~3 min on a busy machine), which forces bypasses — worth folding the timeout into the same cleanup.

[nnunley]

Follow-up on the roadmap references above: the two internal epics are now public — #574 (IR normalization + execution-form cleanup) and #575 (block interface / locals / block-arg erasure). The ops.lg claim, the RPN-spike hold, and the generated.lg retire-don't-rehabilitate call all trace to those.