JVM Command Line Options for Low Latency Applications

In this article, I elaborate Java Hotspot JVM command line options for low latency applications.

Garbage Collection (GC)

When tuning JVM garbage collection following principles should be taken into consideration:

• Minor GC Reclaim Principle: Maximize the number of objects reclaimed in each minor GC.  This will reduce number of full garbage collections and its frequency.
• GC Maximize Memory Principle: The larger the java heap size, the better garbage collector and application performance.
• Constraints: Tune JVM garbage collector based on two of following performance attributes: throughput, footprint and latency.

Minimum JVM command line options for GC are as follows:

-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-Xloggc:

Calendar date and time stamp of GC activities to link application’s own logs.

-XX:PrintGCDateStamps

To monitor and analyse pause events arising from VM safe point operations for low latency applications:

-XX:+PrintGCApplicationStoppedTime
-XX:+PrintGCApplicationConcurrentTime
-XX:+PrintSafepointStatistics

Note, safe point operations can happen due to many reasons, not only GC (for example, bias unlocking/revoking, deoptimization, thread stopping/resuming, exit).  So analysing reasons behind stop world pause times is very important.

Java Heap Size

Amount of data in java heap after a full garbage collection when application in a steady state is the size of live data. Live data size determines size of heap sections as follows:

• Overall Heap size (-Xms and -Xmx) should be 3 or 4 times of live data size
• Permanent Generation (-XX:PermSize, -XX:MaxPermSize) size should be  1.2 to 1.5x times of permanent generation space occupancy.
• Young generation (-Xmn) size should be bigger than 1.5 times of live data size and not less than 10 % heap size.  Size of young generation space should be optimum to reduce frequency and duration of minor GC. For example, if minor GC happens very frequently size of young generation should be increased. Whereas, if duration of minor GC is so high, size of young generation should be decreased.
• Old generation (Heap size - Young generation) size should be 2 to 3 times of live data size.

Initial heap size (-Xms) and maximum heap size (-Xmx) should be same for low latency applications in order to avoid dynamic sizing of heap. This is true for young generation too therefore size of young generation should be specified with flag –Xmn (max and initial size).

In Concurrent Garbage collector (CMS) (-XX:+UseConcMarkSweepGC), special attention should be given to survivor ratio (-XX:SurvivorRatio) and tenuring threshold.

Survivor ratio (-XX:SurvivorRatio) is the ratio  of eden space to survivor spaces. It must be greater than 0 and it is used to determine size of survivor and eden space compare to overall young generation size such as:

survivor space size = -Xmn/(-XX:SurvivorRatio= + 2)

For example if -XX:SurvivorRatio=6 and –Xmn=512m is specified, survivor size will be 64m and eden size will be 384m.

As explained above, since Survivor ratio (-XX:SurvivorRatio) determines size of eden space, so the smaller eden space, the higher frequency of minor GC, which leads short minor GC duration time and more objects are promoted to survivor space. When objects promoted from eden space to survivor spaces, age of objects are increased too.  Objects older than an age (tenuring threshold) are promoted to old generation. Therefore, survivor ratio (-XX:SurvivorRatio) affects tenuring of objects.

Tenuring threshold is calculated by JVM dynamically by analysing aging activity of objects. But -XX:MaxTenuringThreshold  (between 0 and 31 in java 6) can be used to specify tenuring threshold. Objects older than this value are promoted to old generation space.  Hence, an optimum tenuring threshold must be used to prevent promoting objects to old generation.

Tuniring activity can be monitored by enabling -XX:+PrintTenuringDistribution which prints out statistics for each age:

Desired survivor size 8388608 bytes, new threshold 1 (max 15)
- age 1: 16690480 bytes, 16690480 total

In this output it should be analysed in a way that if the number of bytes surviving at each object age decreases as the object age increases and whether the tenuring threshold calculated by the JVM is equal to or stays close to the value set for the max tenuring threshold.  For example, similar to previous example, if internal threshold is less then max tenuring threshold and desired survivor space less than aged byte consistently, then survivor space should be increased.

Another point in CMS garbage collector is also when to initialize the cycle. Following command line options are used to control when CMS garbage collection should be started based on occupancy of old generation space:

-XX:CMSInitiatingOccupancyFraction=
-XX:+UseCMSInitiatingOccupancyOnly

An optimum value of -XX:+UseCMSInitiatingOccupancyOnly should be used minimize frequency of CMS and duration and risk of stop the all world garbage collection. If a high value of -XX:+UseCMSInitiatingOccupancyOnly is used, CMS duration would be longer, also there is a chance that old generation will be full before CMS reclaim space, if application generates more objects than reclaimed.

Explicit Garbage Collections

Unless it is required specifically in the application, garbage collection arising by System.gc() should be disabled or executed as concurrent GC with following options:

-XX:+DisableExplicitGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses

Aggressive Optimization Techniques

Many new and slightly risky optimization techniques can be enabled by -XX:+AggressiveOpts options. This option is not recommended for application in which, stability is important, as this feature enable new optimization techniques which are not fully proven stable yet.

Biased Locking

Hotspot Java 6 has a new default feature which bias locking for towards last thread holding the lock. This approach is true for many applications. But in some cases, that can not be true. For example, a resource locked by producer and consumer threads.  Since revoking a  biased locking requires a full stop the world safe operation, it is better to disable that future for that sort of application with -XX:-UseBiasedLocking option. Statistics regarding to stop the world time and activities can be monitored by following options

-XX:+PrintGCApplicationStoppedTime
-XX:+PrintGCApplicationConcurrentTime
-XX:+PrintSafepointStatistics

Example JVM parameters:

$JAVA_HOME/bin/java –Xmx10G –Xms10G -Xmn1200m -XX:SurvivorRatio=4 -XX:PermSize=128m -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=1000 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=90 -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:gc.log -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:+PrintSafepointStatistics -XX:+AggressiveOpts -XX:+UseCompressedOops -XX:+OptimizeStringConcat -XX:+UseStringCache -XX:+UseFastAccessorMethods

Commands to analyse heap

Getting histogram of objects in JVM heap

PID= |Process id of a java process|
FILE_NAME_HIST=heapHist.dump
FILE_NAME_HIST_LIVE=heapHist_Live.dump 

%% All objects 
$JAVA_HOME/bin/jmap -histo $pid > $FILE_NAME_HIST
%% Live objects 
$JAVA_HOME/bin/jmap -histo:live $pid > $FILE_NAME_HIST_LIVE 

Getting a memory dump of a java process:

pid= |Process id of a java process| 
FILE_NAME_LIVE=heapDump_Live_$pid.dump
FILE_NAME=heapDump_$pid.dump 

%% dump everything in heap to $FILE_NAME 
$JAVA_HOME/bin/jmap  -F -dump:format=b,file=$FILE_NAME $pid 
%%For only Live objects, we need to do following. Note this will force a full GC first, therefore first try above dump. 
$JAVA_HOME/bin/jmap  -F -dump:live,format=b,file=$FILE_NAME_LIVE $pid 

Starting jstatd (to connect a remote java process via visualVM)

# start jstatd (with same user name/functional id)
# an example of tools.policy file is provided below
jstatd  -J-Djava.security.policy=tools.policy  &

# example of tools.policy file
grant codebase "file:${java.home}/../lib/tools.jar" {
   permission java.security.AllPermission;
};