-
Notifications
You must be signed in to change notification settings - Fork 1
Memory Size
One way to reduce Clojure's memory usage
export _JAVA_OPTIONS="-Xmx2g"
My development machine is well endowed with a fast processor, SSD drive, and 32GB of RAM. Nonetheless, it would sometimes grind to a halt when I worked on a ClojureScript project.
The reason was (as is usually the case) disk thrashing. My memory was over-commited. But why?
My machine runs Ubuntu, and typically has running:
- Chrome, with many open tabs.
- A VM running Windows 10.
- Emacs, with many source code buffers, Cider, and two REPLs (one for Clojure and one for ClojureScript.
- A few terminal shells, sometimes running another REPL (e.g., for tests or CSS compilation).
Clojure runs in the Java Virtual Machine (JVM). In my dev setup, I always have at least two JVMs running; sometimes more.
The JVM is a garbage-collected system, of course. (At the risk of over-simplifying quite a bit), it creates objects until it hits its VM allocation is full, and then it garbage collects.
The default tuning is that the JVM will use 1/4 of the machine's physical memory size as its VM maximum. This is a fine solution normally: generous enough to run well, without being too much of a pig.
But, with multiple JVMs running in parallel, things get worse.
In my case, I had allocated 8GB of memory to the Windows VM. Chrome also typically used 8+GB. With a total of 32GB, the two JVMs pushed me right to the edge. With a third JVM, or a briefly greedy Chrome tab, or a busy Windows VM, or all of the above, my machine would start to thrash. It would often freeze for many seconds at a time.
The JVM can be started with a flag -Xmx.. specifying the maximum
virtual memory to use. (Xmx512m, Xmx2g, etc). In my case, 2GB for
each JVM seems like a safe guess: a tiny fraction of my full machine, but
very generous space for my small Clojure projects.
In Leiningen, it is easy to set this option. In project.clj, you can
specify `:jvm-opts ["-Xmx2g"]. In principle, that should be all that was
needed. But, I hit two snags:
- I deploy some of my projects to very small AWS instances. 2GB is too large, and I don't want to change my project.clj for each deployment.
- Even on my dev machine, this setting only changed one of the two JVMs.
Sadly, I don't know exactly how my tooling (Emacs + Cider + Figwheel +
whatever else behind the curtains) fully works, but presumably, the
JVM that reads
project.clj(and, thus, must start before it sees the option) was still running one of the two REPLS.
So, for both reasons, the right thing is to set -Xmx on machine in a
way that will be visible to each of my JVMs.
Easy:
In Linux, I add to my .bashrc
export _JAVA_OPTIONS="-Xmx2g"
Presumably, though I've not tested, on Windows one could do:
set _JAVA_OPTIONS="-Xmx2g"
Unfortunately, this does not quite work. When I start Emacs from my
desktop, it does not read .bashrc. I could start emacs from a terminal
shell. This works; but yuck, it's not my normal workflow.
I found a smoother solution, using the Emacs library exec-path-from-shell to read the shell environment.
Install the package, and then:
~/.bashrc
export _JAVA_OPTIONS="-Xmx2g"
~/.emacs.d/init.sh
(exec-path-from-shell-copy-env "_JAVA_OPTIONS")
(when (memq window-system '(mac ns x))
(exec-path-from-shell-initialize))
There are other, similar, solutions. See a recent discussion on the slack channels. Read starting from "My machine gets extremely slow" on this page of the archives.