You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#655 adds the pinned-release third leg to benchmark/run.sh plus a per-workload regression check, with BASELINE_STRICT=1 to turn it into a non-zero exit. It touches .gitignore, benchmark/BASELINE_REF, and benchmark/run.sh — no workflow — so nothing calls it.
benchmark/ therefore still never executes in CI. No workflow references it, and no Go-benchmark lane can reach it: the directory holds .lg, .clj, and run.sh. The PR lanes (perf-pr, perf-wasm) run the pr-fast profile, a single pkg/vm job with a fixed family regex, and perf-timeline runs -full, covering pkg/vm and test. None of them execute a let-go program.
This issue is the lane that calls #655's check. The rationale for the pinned baseline itself lives there and is not repeated here.
Proposal
A workflow following the existing perf-*.yml shape:
Label-opt-in, matching the perf / perf-repeat / perf-wasm pattern, so unlabeled PRs cost nothing.
CI caught neither. Both were found by hand, two days and one week after their respective merges. #655 makes the reduce case visible to the harness; a lane is what puts it in front of someone without running the suite locally.
Open questions
Obtaining and caching the pinned lg.go install at a tag is simplest; building from a tag in a second checkout is more faithful for the AOT leg. Either wants caching, since this binary never changes by design.
Runtime budget. Seven workloads across three legs under hyperfine. perf-pr-repeat already carries a 90-minute cap for a smaller matrix.
Whether startup belongs in the lane. It is milliseconds and noise-dominated, so it may fit an always-on check better than a delta table.
Summary
#655 adds the pinned-release third leg to
benchmark/run.shplus a per-workload regression check, withBASELINE_STRICT=1to turn it into a non-zero exit. It touches.gitignore,benchmark/BASELINE_REF, andbenchmark/run.sh— no workflow — so nothing calls it.benchmark/therefore still never executes in CI. No workflow references it, and no Go-benchmark lane can reach it: the directory holds.lg,.clj, andrun.sh. The PR lanes (perf-pr,perf-wasm) run thepr-fastprofile, a singlepkg/vmjob with a fixed family regex, andperf-timelineruns-full, coveringpkg/vmandtest. None of them execute a let-go program.This issue is the lane that calls #655's check. The rationale for the pinned baseline itself lives there and is not repeated here.
Proposal
A workflow following the existing
perf-*.ymlshape:perf/perf-repeat/perf-wasmpattern, so unlabeled PRs cost nothing.benchmark/run.shwithBASELINE_STRICT=1against theBASELINE_REFbench: compare against a pinned-release baseline #655 pins.lgversion, so a delta is attributable to a lane and not to an unlabeled runner.perf-pr-repeatran during its shadow phase.Why the lane is worth adding
Two regressions found in the past week live in workloads the suite already covers:
reduceis 1.75x slower than v1.12.2 — bisected to #639 #656 —reduce1.88x slower than v1.12.2 (measured 2026-08-05), fixed in perf(rt): restore reduce's ArrayVector and Range fast paths #686.benchmark/reduce.cljexists.Frame.Runis Go-recursive — deep lg→lg recursion aborts the process #644/vm: make direct bytecode calls non-recursive with an explicit frame chain #645 — bytecode invoke roughly doubled, worth ~1.4x onsome/reduce/mapshapes.transducersandmap-filterexercise that path.CI caught neither. Both were found by hand, two days and one week after their respective merges. #655 makes the
reducecase visible to the harness; a lane is what puts it in front of someone without running the suite locally.Open questions
lg.go installat a tag is simplest; building from a tag in a second checkout is more faithful for the AOT leg. Either wants caching, since this binary never changes by design.hyperfine.perf-pr-repeatalready carries a 90-minute cap for a smaller matrix.startupbelongs in the lane. It is milliseconds and noise-dominated, so it may fit an always-on check better than a delta table.