Tuning JVMs

At work I was dealing with the only group operating a JVM at the company, and none of the other engineers had a background with that platform. This led me to distill, not the details about all the various options, but a high level guide to how to think about tuning them.

tl;dr: Set -Xmx (max heap size) always. Don’t use any other memory or GC tuning options unless you have verified that it is important for your workload and have a paper trail leading from the option to why it’s set (e.g., each option is set on its own line in a config file under source control, and the diff it was introduced in links to the ticket with the investigation, etc.).

It’s useful to think of JVM memory in three pieces:

The heap is controlled by two options: -Xmx (max heap size) and -Xms (startup heap size).

-Xmx is important, and needs to get set on basically any Java service you run or most of your machine’s memory will sit unused while the JVM thrashes in its limited heap size.

-Xms tells the JVM to request this much memory from the kernel for the heap in advance when it starts up rather than asking for it on demand. If you have a program that starts up, loads a bunch of data, does something to it, and shuts down, this can give you some performance boost. In a long running server, once it warms up it’s going to request enough heap anyway.

The stack has one important option: -Xss (max per-thread stack size)

-Xss limits how big the stack for each thread can get. Unless you’re doing numeric computation in an enormous, recursive algorithm, you almost certainly don’t need this. 50 64 bit local variables per frame in a call stack one hundred deep is still ~50kb.

We don’t really get to control the “static” memory. If you want less there, load less code.

GC options vary from garbage collector to garbage collector. Over the years, the JVM developers have tried to make the default option better and better.

It can be tempting if you try tuning one of these settings to say, “Eh, it made no difference, just leave it.” But say you’re at a company with thousands of different workloads on Java. You open up a service, look at its JVM config, and find all these options set, including an old GC chosen and tuned. Then you go to a different service and find some subset of them. And another service has a different subset. Are these important? Should you mess with them?

The paths to sanity are either

  1. have a dedicated team that tunes the JVM for everyone and no one sets their own options (see Java EE app servers), or
  2. set only `-Xmx``, and add any others one at a time with a paper trail explaining why