JVM tuning or premature optimization?

JVM tuning is sometimes a big topic on projects. Depending on the experience people made with previous projects they start to apply a whole bunch of parameters immediately to the JVM, before they even run the application the first time.Or they just don’t know, that there are parameters beyond -Xmx and -Xms. So, what’s the way you should go?  Before I will give my personal recommendation about it, let me explain a few things first.

JVM tuning (which is in most cases only “Garbage Collection optimization”) is a horribly complex topic for a few reasons:

•  All major JVMs (Sun/Oracle Java, IBM Java, and let’s also add OpenJDK) have emerged over a long time, and many many man-years of research went into it, to make it efficient. Efficient to an extent, that it sometimes even beats compiled code. For this the engineers implemented hotspot-compilation, code-analyzers and optimizers and lot of interesting stuff more. Also the garbage collection algorithms improved vastly over time. In the end I don’t think, that anyone outside the core engineering team of this JVM understands the full impact of the more complicated JVM switches.
• Most JVMs have a lot of auto-tuning parameters, which adapt itself to a certain extent to the application. A lot of heuristics & and statistics has been built-in to avoid major tunings. (Of course if these auto-tuning is working against you, you should start tweaking parameters yourself.)
• When switching over to a new version of the JVM, no one guarantees, that the set of JVM parameters you have applied from the previous version will continue to work smoothly. In most cases these switches are not really good documented (probably by purpose) and they are not guaranteed to be effective.
• And especially in the context of garbage collection tuning: You tune for a specific application and for its specific behavior. Some applications have a lot of long-living objects and have high concurrency in accessing these objects. Others create a lot of short-living objects in a thread and dispose them after some task has been performed and create no or a very small number of long-living objects. For these 2 extreme scenarios your garbage collection will behave completely different and when you want or need to optimize it, you need to optimize differently and also for different goals. In a batch-scenario you will optimize for high throughput while in an interactive scenario you optimize for low latency (and sacrifice a little bit throughput for it).

So, just because you have a proven set of parameters from a different project, you cannot blindly apply it. In the best case it might help a little bit, but you don’t know why. In the normal case, these switches just don’t have a measurable impact on performance. And in the worst case, you degrade your performance, and you also don’t know why (but you have the feeling, that you have done it right).

So, what are my recommendations for JVM tuning for CQ5 applications:

1.  Do not try to optimize the JVM at all. In the majority of CQ installations I’ve worked with no optimization has been performed and the JVM behaved very well. Analysis has shown no problems in the JVM and garbage collection, so there was no need to modify here.
2. Have enough information to analyze in case you have a problem. I usually add a few informational parameters to the JVM parameters, like garbage collection logging, creating a heap dump on out of memory and things like that. These are not tuning parameters in the first sense, but give me more information about the JVM itself. Normally I don’t need them, but in case of problems it is good to have these information.
3. And if you have a problem you track down the JVM itself, ask an expert on that topic. Just googling and blindly adding parameters to the JVM isn’t helpful, if you don’t know, what this parameter changes at all and if this is relevant for you. And then don’t forget to prove that your change has a positive effect.

My recommended JVM parameters for Oracle JVM (1.6) besides the default ones specified by the CQ start script:

• -XX:+HeapDumpOnOutOfMemoryError — write an heapdump if the JVM runs out of heap space
• -XX:+HeapDumpOnCtrlBreak — on windows create an heapdump when pressing ctrl+break
• -verbose:gc — verbose garbage collection logging
• -XX:+PrintGCTimeStamps — print timestamps in the GC log
• -XX:+PrintGCDetails — even more GC information
• -Xloggc:gc.log — write the GC logs to this file and not to stdout
• -XX:+UseGCLogFileRoatation (since Java 6u32 or Java 7u2) — enable GC log rotation
• -XX:+NumberOfGCLogFiles=10 (since Java 6u32 or Java 7u2) — and keep 10 versions of that GC log file

To understand how the memory model of modern JVMs work (especially the generational heap model of the Oracle JVM), please read this great article on Java Garbage Collection.

While I appreciate the work my colleague Jayan Kandathil did (he posted some settings he benchmarked to be most effective for his specific type of workload), I would not recommend that you just blindly apply these settings on your own instance. Because very likely your workload will be a bit different.