diff --git a/README.md b/README.md index a3aa534af..d08972cc7 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ Greetings loafers! _(λ-gophers haha, get it?)_ let-go is a Clojure dialect with a bytecode compiler and stack VM, written in Go. -A single ~12MB binary, ~8ms cold start, no JVM. It passes the +A single ~13MB binary, ~10ms cold start, no JVM. It passes the [jank-lang test suite](https://github.com/jank-lang/clojure-test-suite). I started this in 2021 as an elaborate joke: an excuse to write Clojure while @@ -42,25 +42,29 @@ Non-goals: drop-in JVM Clojure replacement; linter/formatter for Clojure-at-larg ## Benchmarks -let-go vs Babashka, Joker, [go-joker](https://github.com/rcarmo/go-joker), -[gloat](https://github.com/gloathub/gloat), and Clojure JVM. All benchmark -files are valid Clojure that runs unmodified. Apple M1 Pro. +let-go (bytecode VM), let-go AOT (the same code IR-lowered to native Go), +Babashka, and Clojure JVM. All benchmark files are valid Clojure that runs +unmodified; the VM and AOT legs run the identical program — only dispatch +differs. Apple M1 Pro. -| | let-go | babashka | joker | go-joker | gloat | clojure JVM | -| --------------- | ---------- | -------- | ----- | -------- | ----- | ----------- | -| **Binary size** | **12MB** | 68MB | 26MB | 32MB | 26MB | 304MB (JDK) | -| **Startup** | **8.2ms** | 17.7ms | 11.5ms | 12.5ms | 14.7ms | 360ms | -| **Idle memory** | **14.7MB** | 27.0MB | 21.6MB | 23.7MB | 22.9MB | 98.0MB | +| | let-go | let-go AOT | babashka | clojure JVM | +| --------------- | ---------- | ---------- | -------- | ----------- | +| **Binary size** | **13MB** | 18MB | 68MB | 304MB (JDK) | +| **Startup** | 11.1ms | **10.7ms** | 20.4ms | 364ms | +| **Idle memory** | **15.2MB** | 15.2MB | 27.0MB | 97.7MB | let-go stays compact and quick to launch: a small native binary, fastest startup in this run, low RSS, and no JVM dependency. -On runtime benchmarks, let-go is competitive on short-lived data work like -map/filter (7.2ms) and persistent maps (20.2ms), and it is still much faster -than upstream Joker on numeric/tree-walk-heavy cases. go-joker's WASM JIT leads -hot loops and reduction/transducer workloads; Babashka is ahead on several -algorithmic cases; and the JVM dominates long compute runs once HotSpot warms -up. +On runtime benchmarks, the VM is competitive on short-lived data work — +map/filter (11.3ms) comes in 1.7× ahead of Babashka, and persistent maps +(22.2ms) run neck and neck with it — while AOT lowering turns the call-heavy +numeric cases around entirely: fib(35) drops from 2.42s to 0.11s and tak from +2.40s to 94ms, an order of magnitude ahead of Babashka. Fully-warmed HotSpot +still holds the raw-compute crown (steady-state JVM fib(35) is ~76ms to AOT's +~99ms, measured in-process) but pays ~360ms of startup first, so on one-shot +runs let-go AOT delivers the result ~5× sooner. Babashka still leads +reduction/transducer workloads. Full per-benchmark numbers and methodology: [benchmark/results.md](benchmark/results.md). diff --git a/benchmark/results.md b/benchmark/results.md index f9d3dd3a4..a11999320 100644 --- a/benchmark/results.md +++ b/benchmark/results.md @@ -30,28 +30,28 @@ Clojure JVM times include full JVM startup (~350-500ms) which dominates short be | Runtime | Time | |---|---| -| let-go | 9.1ms ± 1.2ms (1.0x) | -| **let-go AOT** | **8.7ms ± 0.8ms** (1.0x) | -| babashka | 20.9ms ± 1.9ms (2.3x) | -| clojure JVM | 0.362s ± 0.018s (39.8x) | +| let-go | 11.1ms ± 0.8ms (1.0x) | +| **let-go AOT** | **10.7ms ± 1.1ms** (1.0x) | +| babashka | 20.4ms ± 1.3ms (1.8x) | +| clojure JVM | 0.364s ± 0.010s (32.8x) | ### Peak Memory Usage (RSS) | Workload | let-go | let-go AOT | babashka | clojure JVM | |---|---|---|---|---| -| startup (nil) | **15.0MB** (1.0x) | 15.2MB (1.0x) | 27.0MB (1.8x) | 103.4MB (6.9x) | -| fib(35) | 16.0MB (1.0x) | **15.6MB** (1.0x) | 77.4MB (4.8x) | 121.8MB (7.6x) | -| reduce 1M | **21.0MB** (1.0x) | 22.0MB (1.0x) | 59.2MB (2.8x) | 121.9MB (5.8x) | +| startup (nil) | **15.2MB** (1.0x) | **15.2MB** (1.0x) | 27.0MB (1.8x) | 97.7MB (6.4x) | +| fib(35) | 15.6MB (1.0x) | **15.5MB** (1.0x) | 77.4MB (5.0x) | 119.0MB (7.6x) | +| reduce 1M | 21.7MB (1.0x) | **21.4MB** (1.0x) | 59.2MB (2.7x) | 116.8MB (5.4x) | ### Performance | Benchmark | let-go | let-go AOT | babashka | clojure JVM | |---|---|---|---|---| -| fib | 2.424s ± 0.028s (1.0x) | **0.113s ± 0.001s** (0.0x) | 1.935s ± 0.021s (0.8x) | 0.603s ± 0.018s (0.2x) | -| loop-recur | 70.9ms ± 0.9ms (1.0x) | **10.6ms ± 0.8ms** (0.1x) | 69.0ms ± 1.9ms (1.0x) | 0.501s ± 0.015s (7.1x) | -| map-filter | **10.7ms ± 1.7ms** (1.0x) | 12.4ms ± 1.7ms (1.2x) | 22.7ms ± 3.6ms (2.1x) | 0.398s ± 0.015s (37.1x) | -| persistent-map | **23.0ms ± 1.3ms** (1.0x) | 24.0ms ± 1.1ms (1.0x) | 29.3ms ± 6.1ms (1.3x) | 0.534s ± 0.017s (23.2x) | -| reduce | 41.0ms ± 1.5ms (1.0x) | 68.2ms ± 3.5ms (1.7x) | **37.6ms ± 2.3ms** (0.9x) | 0.376s ± 0.022s (9.2x) | -| tak | 2.432s ± 0.025s (1.0x) | **94.4ms ± 1.1ms** (0.0x) | 1.939s ± 0.022s (0.8x) | 0.649s ± 0.011s (0.3x) | -| transducers | 60.9ms ± 21.3ms (1.0x) | 49.4ms ± 1.6ms (0.8x) | **30.8ms ± 2.4ms** (0.5x) | 0.431s ± 0.014s (7.1x) | +| fib | 2.416s ± 0.012s (1.0x) | **0.110s ± 0.001s** (0.0x) | 1.943s ± 0.049s (0.8x) | 0.567s ± 0.005s (0.2x) | +| loop-recur | 68.8ms ± 1.0ms (1.0x) | **10.7ms ± 0.9ms** (0.2x) | 64.4ms ± 1.3ms (0.9x) | 0.475s ± 0.004s (6.9x) | +| map-filter | 11.3ms ± 0.8ms (1.0x) | **10.4ms ± 0.7ms** (0.9x) | 19.2ms ± 0.9ms (1.7x) | 0.387s ± 0.011s (34.3x) | +| persistent-map | 22.2ms ± 0.7ms (1.0x) | 22.7ms ± 1.4ms (1.0x) | **21.7ms ± 1.2ms** (1.0x) | 0.517s ± 0.011s (23.3x) | +| reduce | 39.3ms ± 1.1ms (1.0x) | 66.6ms ± 1.3ms (1.7x) | **36.0ms ± 1.0ms** (0.9x) | 0.390s ± 0.015s (9.9x) | +| tak | 2.397s ± 0.022s (1.0x) | **93.7ms ± 0.6ms** (0.0x) | 1.908s ± 0.037s (0.8x) | 0.613s ± 0.046s (0.3x) | +| transducers | 46.5ms ± 0.9ms (1.0x) | 43.0ms ± 1.0ms (0.9x) | **20.3ms ± 0.5ms** (0.4x) | 0.373s ± 0.011s (8.0x) |