Tuning the Java Virtual Machine
All Java applications rely on a Java Virtual Machine (JVM) to execute and making sure your JVM is running as efficiently as possible is important to make sure that your ColdFusion MX 7 applications are working as efficiently as possible as well. In the next sections we are going to look at how the JVM works, how to Garbage Collection works, how to change and control the JVM, and how to monitor the JVM via log files and by visualizing the JVM so that you can tune the JVM for optimal performance.
Introducing the JVM
A JVM is an implementation of Sun's Java Virtual Machine specification, which defines an abstract machine or processor that is basically a software version of a hardware processor or CPU. The JVM spec also specifies an instruction set, a set of registers, a stack, a "garbage heap," and what is called a method area. Software companies (including Microsoft and IBM), vendors, and others create JVMs for specific platforms, although by far the most popular is Sun's JVM. Once a JVM has been created for a given platform, any Java program (compiled Java source, or bytecode) can run on that platform. A JVM can either interpret bytecode one instruction at a time, mapping it to a real processor instruction; or it can further compile the bytecode for the real processor, using what is called a just in time (JIT) compiler, which radically improves performance.NOTEFor a more detailed look at the Sun JVM specification, look at http://java.sun.com/docs/books/vmspec/.The JVM is the heart of any Java application and so must work as efficiently as possible. One of the main reasons a JVM may perform poorly is that it's not using memory efficiently. In general, the JVM is very efficient, but it can use some help in high-performance applications where it's expected to do a lot of work, manage a large number of objects, deal with large data sets, or simply operate as fast as possible. In this section we focus on some things you can do make your JVM excel for your specific application.ColdFusion MX 7 ships with a series of default JVM attributes that are designed to work on almost any system or setup but are not the best configuration for every application. This is also true for almost every other J2EE application server, and knowledgeable J2EE system managers routinely profile and tune their J2EE servers based on their particular system and application operation parameters. When tuning your JVM, the first place to start is to figure out what the heap value for your JVM will be. Java applications make use of a garbage collection heap, which is a specialized memory structure. "The heap" is where all the objects of a Java program live. This means every ColdFusion page, CFC, session variable, and so on that becomes a Java object resides in the heap.The JVM periodically performs "garbage collection" to remove or de-allocate unused Java objects from the heap. Your heap value and settings determine in part how often and for how long garbage collection occurs. These settings are very important because, while garbage collection is happening, the JVM essentially pauses and puts all other operations and requests on hold. Usually garbage collection takes so little time and is so efficient that it has relatively no impact on performance, but in certain instances where there are few resources or the system is under high load, performance can take a hit during garbage collection.If the heap value for your JVM is too low, you may see such things as the "Java.lang.OutOfMemory" error, ColdFusion server timeouts, poor response times under load, and what appear to be memory leaks. Furthermore, garbage collection may occur so often that it affects performance, which also leads to similar issues. Numerous other things unrelated to JVM tuning can cause these behaviors, but often the reason is that the application is demanding more from the JVM than it can deliver with resources it has.ColdFusion's default setting for the size of the JVM heap is 512 MB. (The JVM settings, jvm.config, are in server_root\bin.) This is often sufficient for most applications, but if your application has a large amount of traffic, makes lots of large database queries, or otherwise requires more heap space, then you will want to set it higher. The easiest way to tell what your heap should be is by profiling your application using a product such as OptimizeIT (http://www.borland.com/optimizeit/) or JProfile (http://www.ej-technologies.com/). These programs help you easily determine the amount of memory your application is using and how it is using it. Another method is to use a performance monitoring tool, such as perfmon on Windows, and monitor the application under load. This way you can get a sense of memory usage even at peak times. If you find that the system is often using all the memory available to it, then you'll want to increase what you allocate to the heap.Generally, if your server has substantial free memory, you'll want to assign it to the heap. A server with 2 GB memory can afford a JVM heap maximum setting of 1000 MB. A good rule is to assign the JVM as much free memory as possible without causing the system to swap pages to disk. On production systems, you should also set the JVM heap's minimum to be the same as the maximum. This reduces periodic pauses of the JVM as it allocates more memory to itself, up to the maximum setting, and reduces what is called heap fragmentation.So to set the maximum and minimum heap to 1 GB, you'd edit the jvm.config for each server instance that is running ColdFusion. On your J2EE server, look for # Arguments to VM and edit:
to read:
java.args=-server -Xmx512m
You don't want to touch the text java.args=-server, which is what Java expects to see to let it know that everything after that attribute comprises settings for the JVM. The -Xmx stands for heap maximum, and the -Xms stands for heap minimum.TIP
java.args=-server -Xmx1000m -Xms1000m
Since the JVM is memory intensive, and memory is one of the largest bottlenecks for any Java application, always try to reasonably maximize memory on your servers. Memory is cheap, and having at least a gigabyte of memory available to your JVM heap will make a significant positive impact on performance.NOTEJVM tuning may not be rocket science, but it's not trivial, either. If you want more information about tuning Sun's JVM 1.4+, check out http://java.sun.com/docs/hotspot/gc1.4.2/. If you use a different JVM, read the vendor's documentation. You should also read the official Sun JVM Tuning FAQ here: http://java.sun.com/docs/hotspot/gc1.4.2/fa230.
Tuning Garbage Collection
Now that we have discussed what the JVM is and how to set the heap size, let's take a look at how to tune garbage collection. Generally the JVM does a good job of cleaning up all the unused objects in the heap, but if you have profiled your application, you can optimize the JVM for the specific application. In this section we'll describe some of the most common options and how to set them in jvm.config. Then we'll see how you can use the jvmstat tool and visualgc to help you determine what settings you will want to use for your specific application's JVM.TIPIncorrectly setting JVM attributes or even a simple typo can cause your J2EE server to fail to start. Simply replacing the jvm.config file with one that has the original settings will resolve this, so we suggest that you keep a copy of the original settings. Or, better yet, keep your jvm.config in a version control system with updates and notes every time you change the file.Before we start discussing specific garbage collection options, let's take a closer look at heap memory usage and garbage collection. In this section, you'll encounter some special Java terminology that will be important not only for understanding the garbage collection but also the garbage collection log files. If you plan to read more about tuning the JVM, you'll need an understanding of these terms.The JVM breaks up the heap into pools or buckets of memory, into which are assigned Java objects that have specific life expectancies; after these periods of time end, the objects need to be cleaned up. These memory buckets are called generations, based on the life expectancy of their occupants. There are three major generations: young, tenured , and perm (for permanent, which is actually a special type of tenured generation). Objects that reside in the young generation are typically only going to last for, say, the life cycle of a ColdFusion page request. Objects with longer life expectancy, such as a cached query or even the classes that make up ColdFusion itself, would reside in the tenured generation. The perm generation contains objects that make up the actual JVM. Figure 4.13 will give you an idea of the heap's generations.
Figure 4.13. The heap is divided into the young, tenured, and permanent generations.

Finally, it's important to mention the permanent generation. Some applications, including ColdFusion MX 7, dynamically create and load a hefty quantity of classes into memory. With large ColdFusion applications, this can be an issue. This is often seen in situations where the "Java.lang.OutOfMemory" error crops up in your log files and can be easily solved by increasing the maximum permanent generation (MaxPermSize). For example, to increase maximum permanent generation size to 256 MB, add this to the JVM arguments:
-XX:NewSize=100m -XX:NewMaxSize=100m
-XX:MaxPermSize=256m
Selecting a Collector
The next most important thing after deciding on the size of your young generation is determining what garbage collection method to use. Generally, the JVM's default collector is fineespecially if your server has a single CPUbut other collectors can offer performance increases in these situations, as well as on servers that have more than one CPU. There are three different types of collectors: throughput, concurrent, and incremental.The throughput collector is set by adding the string:
to the JVM arguments string. This collector is a parallel version of the default young generation collector and is especially useful on servers with many CPUs and a large young generation.TIPThe throughput collector is especially powerful on systems with many processors because, unlike the default collector, you can have the throughput collector use multiple threads for garbage collection. You can do this by adding the string-XX:ParallelGCThreads=<desired number> after -XX:+UseParallelGC, where desired number is usually the number of CPUs on the server.The concurrent collector is set by adding the string:
-XX:+UseParallelGC
after the JVM arguments string. This one is often called the "low pause" collector, in that it pauses the application for very short periods while it does collection on the tenured generation. You can also add a parallel version of the young generation collector, by adding this combination of parameters:
-XX:+UseConcMarkSweepGC
to the JVM arguments string. If you decide to use this option, you'll need to delete the -XX:+UseParallelGC collector; this is usually the default collector that ships with ColdFusion's jvm.config file.Finally, there is the incremental collector, which performs many small collections on the tenured generation but in most cases is slower than the default collection setting. You can set the incremental collector by adding:
-XX:+UseConcMarkSweepGC -XX:+ParNewGC
to the JVM arguments string.All these collectors give you alternatives to the ColdFusion's default collector, but changing the default collector is the last thing you should attemptonly after increasing your heap size and tuning your young generation. After you have made those changes, you can try changing the collector to see if any of the optional collectors will increase performance.
-Xincgc
Testing and Tuning
Now that you understand how memory is allocated, how to assign memory to the heap and the various generations, and how to assign specific garbage collections methods to the JVM, you should test your application. The simplest way to do this is generally to use your testing suite to put the application under load and monitor the effects of the new settings on application performance. If you don't see increases in throughput, try a different approach.Another approach, especially if you are seeing intermittent errors or performance issues, is to dump the output of garbage collection to your logs. You do this by simply adding these parameters to your JVM arguments:
-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC
- The -XX:+PrintGCDetails tells ColdFusion to print out garbage collection details.
- The -XX:+PrintGCTimeStamps tells it to add timestamps (which can be really useful in debugging so that you can correlate problems to specific times).
- The -XX:+PrintHeapAtGC will print out detailed information on the generations. By default, ColdFusion will output this data to serverinstance_root\runtime\logs\<servername>-out.log.
If you would like to log the garbage collection data to its own file, you can also add -Xloggc:<filename>, where filename is chosen by you.
- The output from the garbage collection will look something like this:
[View full width]11948.248: [Full GC {Heap before GC invocations=248:
Heap
def new generation total 49088K, used 4735K [0x10010000, 0x13010000, 0x13010000)
eden space 49024K, 9% used [0x10010000, 0x104afda0, 0x12ff0000)
from space 64K, 0% used [0x12ff0000, 0x12ff0000, 0x13000000)
to space 64K, 0% used [0x13000000, 0x13000000, 0x13010000)
concurrent mark-sweep generation total 999424K, used 28683K [0x13010000, 0x50010000,0x50010000)
concurrent-mark-sweep perm gen total 65536K, used 22628K [0x50010000, 0x54010000, 0x58010000)
At first this might be intimidating and hard to read, but look at it carefully and it will start to make sense. Let's break it down. The first line marks specific events in seconds from when ColdFusion was started. In this example, a full garbage collection was started. Next comes detailed information about the various generations in the heap. Notice the first one, the new generation, which is the same thing as young generation; it has a total of 49088K allocated to it but is only using 4735K of the memory. The next two lines after the eden space are the from and to, which are essentially the survivor spaces; they are totally empty.The next line shows that the concurrent garbage collection method was used, that the total heap was 999424K (1 gigabyte), and only 28683K was used. The last line shows that the concurrent collector was used on the permanent generation and that its size was 65536K, with only 22628K used.As you can see, the output from the garbage collection is not too difficult to understand but it can be difficult to read and determine what actions you need to take just from look at a few lines in your log files. There are garbage collection log file readers such as HP's HPjtune (http://www.hp.com/products1/unix/java/java2/hpjtune/212), a free tool that reads garbage collection log files you have created and provides a variety of visual reports that may make your job easier. You should look into a tool like this in that you can look at log files from a week of actual usage and get a much better sense of how your system is behaving. Many of the more sophisticated commercial products also have built-in garbage collection analysis tools. If you find yourself often having to tune the JVMs of your application, then an investment in one of these will be well worth the money.Not surprisingly, many ColdFusion developers as well as system administrators would rather not learn the JVM specification by heart. That said, knowing how the JVM functions and how to tune it will greatly increase your ability not just to troubleshoot possible errors and performance problems, but to squeeze the maximum performance from your ColdFusion applications.
Visualizing Garbage Collections
One of the hardest things about tuning the JVM is understanding the information that is being generated in your log files and trying to relate that to your settings and performance. While there are excellent tools such as HP's Jtune for understanding your JVM Garbage collection log files a more intuitive way to tune your JVM is by watching actually seeing how each of the various generations are being used by the JVM while your application is under load. You can do this by using the visualgc tool which was described in chapter 2 Monitoring System performance.If you configure your system to use the visualgc tool and then monitor your application under load you should see something like Figure 4.14. You should see all the JVM attributes set in your jvm.config file as well as graphical information for the tenured space (called Old Gen in visualgc), permanent generation, Eden (Young Generation) and Survivor 0 and Survivor 1. You should also see graphs for all these information as well as information for Garbage Collection which gives the total number of collections and the collective amount of time garbage collection has taken.
Figure 4.14. This figure shows the JVM for a simple ColdFusion MX 7 application while being place under load by a load testing tool.
[View full size image]
