`reduce` is 1.75x slower than v1.12.2 — bisected to #639

[mparrett]

What

(reduce + 0 (range 1000000)) runs 1.75x slower on main than before 428ce9e3 (#639, "hoist 222 clojure.core primitives to named //lg:native decls"). darwin/arm64, go1.26.3, hyperfine 20 runs:

build	mean	GC cycles	CPU / wall
862c7cec (#622, the parent)	33.1 ms ± 0.9	6	33.8 / 33.1
428ce9e3 (#639)	57.7 ms ± 0.8	30	72.5 / 57.7

Allocation rather than compute: five times the GC cycles for identical work, and CPU exceeding wall time where the parent's was flat. Confirmed in both benchmark orders, so it isn't thermal drift.

Blast radius: reduce only

Same two binaries across the whole suite:

workload	ratio
fib	1.03x
tak	1.07x
loop-recur	0.96x
map-filter	0.95x
persistent-map	1.04x
transducers	1.06x
reduce	1.77x

Everything but reduce is inside run-to-run noise.

Plausible mechanism

reduce invokes its reducing fn once per element, a million times in this fixture, through ec.Invoke(mfn, []vm.Value{acc, elem}) in reduceColl (pkg/rt/native_prims.go). + is one of the 222 prims #639 hoisted, so anything the hoist added to a core fn's per-call path gets multiplied by the element count. That matches both the allocation signature and why only reduce moved.

map-filter also calls reduce and is unaffected, which fits: its reduce walks 100 elements, not 1,000,000.

I have the commit, not the line: which of the hoisted prims regressed, and what allocates on each call, are both still open.

Repro

cat > reduce-in-main.clj <<'EOF'
(defn -main [] (reduce + 0 (range 1000000)))
(-main)
EOF

for sha in 862c7cec 428ce9e3; do
  git checkout -f -q $sha && git checkout -q -- . && git clean -xqfd
  make build && cp lg /tmp/lg-$sha
done
hyperfine --warmup 5 --runs 20 "/tmp/lg-862c7cec reduce-in-main.clj" \
                               "/tmp/lg-428ce9e3 reduce-in-main.clj"
GODEBUG=gctrace=1 /tmp/lg-428ce9e3 reduce-in-main.clj 2>&1 | grep -c '^gc '

The git clean -xqfd is required, and -fd alone is not enough. make regenerates pkg/rt/core_compiled.lgb on timestamp rules and git checkout rewrites mtimes, so an incremental build across a checkout can pair a stale core bundle with freshly compiled Go. Those binaries mismeasure by ~1.8x and do it consistently: the error survives repeated runs and survives reversed-order A/B, so it reads as a real regression. Bisecting this without -x produced three different "first bad commits", including one whose diff only touches pkg/api.

Why the benchmark suite didn't catch it

benchmark/run.sh compares the VM leg against the AOT leg, both built from the working tree. A regression in shared runtime code moves both, the ratio between them stays flat, and the suite reports parity. On current main it prints let-go 69.1 ms / let-go AOT 69.1 ms and reports nothing unusual. #655 adds a pinned-release baseline leg so this class of regression is visible; with it, reduce reports 1.66x REGRESSION and the other six workloads stay quiet.

Suggestion

Unreleased, so no user impact yet, but it's on main today, and worth finding which prim regressed before the next release cut. Happy to keep going on it; whoever knows what the hoist changed about native dispatch will likely get there faster.

[mparrett]

Root cause: reduce had two implementations, and #639 changed which one runs. No hoisted primitive regressed.

• pkg/rt/lang.go held a NewCtxNativeFn closure carrying ArrayVector and Range fast paths plus a reused two-element argument buffer.
• pkg/rt/native_prims.go held the //lg:native Reduce/Reduce3 decls delegating to reduceColl, with none of that.

Both predate #639. The generated registration used to land in a distinct "clojure.core" namespace instead of canonical "core" — the alias bug #639's first commit fixed — so it was invisible and the closure won the name. Once the alias resolved, the generated registrar took the var root, because it drains after installLangNS. reduce switched to the naive body with no diff to reduce itself and no failing test.

That body allocates a fresh []vm.Value{acc, elem} per element plus an ArrayChunk per 32 elements via ChunkedFirst: 66.5 MB against 17.5 MB reducing (range 1000000) (measured 2026-08-05), which is the 6 → 30 GC-cycle jump in the table above.

So the suggestion to find which prim regressed is a dead end. There is no single prim, and the 222 hoisted primitives need no audit for this.

Nothing caught it because TestReduceRangeFastPath asserts results, not which path produced them.

Fix

#686 ports both fast paths and the reused buffer into reduceColl and deletes the closure, leaving one implementation: 64.9 → 41.3 ms against pre-#639's 34.6 ms (measured 2026-08-05). Every reduce shape improves, including the general seq path, which never had the buffer.

The residual above pre-#639 belongs to bytecode invoke, which roughly doubled in #645. That measurement is on #644.

Twelve other names are registered both by hand and by the generated registrar. All twelve pairs were compared body for body for #686 and the //lg:native decls are faithful ports, so reduce was the only one that had drifted. #686 adds a recorder plus a ratchet test so a new collision fails CI.

The parity note in the body is why the suite stayed quiet: #655 adds the pinned-release leg that makes this class visible, and #693 is the lane that would run it in CI.

[mparrett]

Fixed by #686, merged as 8a8222d0.

(reduce + 0 (range 1000000)) on darwin/arm64, hyperfine 3 warmup / 14 runs: 64.9 ms on the old main, 41.3 ms after, against 34.6 ms pre-#639. Every reduce shape improves, not only the two with fast paths. #686 also adds ShadowedHandRegistrations() and a ratchet test, so a new duplicate registration fails CI instead of surfacing in a benchmark years later.

Two follow-ups came out of it, both under #531: #696 retires the twelve remaining shadowed hand registrations, #697 drains the rest of pkg/rt's hand registrations onto //lg:native.

The gap between 41.3 and pre-#639's 34.6 is not a reduce problem — it is the general bytecode invoke cost from #645, which this issue's title misattributes to #639. That is now tracked separately in #700, with the measurements, so closing here.