KBA00023001

Different sites have different memory requirements for Tomcat, based on what interfaces they use and how many users they have. As we move more functionality into webapps hosted by both the primary Tomcat server and the Interfaces Server memory requirements will grow and it becomes more likely that you will need to adjust these settings.

This document explains how to adjust the amount of memory which Tomcat and the Interfaces Server use, as well as how to adjust other critical Java Virtual Machine settings.

Throughout this document $CATALINA_HOME is used to represent either the Primary Tomcat's installation directory, or the Interfaces Server's install directory. Each Tomcat server's settings are independent from each other, so please understand that in this document that $CATALINA_HOME points to whichever Tomcat server you happen to be working on.

How will I know that I should adjust Tomcat's memory budget?

Rule of Thumb: If the customer is on 1503 or later their primary Tomcat needs more than 2GB of memory on the heap. Their Interfaces Tomcat may need more than 1GB, but first check the Performance webapp.

Up to the 1503 patch the primary Tomcat has been configured to use 2048 Megabytes (i.e. 2 Gigabytes) of memory. Increasing responsibilities and demands on the server have proven that this amount is too little going forward. However, as of the 1511 patch the code responsible for increasing this allotment has not been working, leaving some servers with too little memory. In worse cases, hand-tuned settings have been overwritten and reset back down to 2GB, leading to downtime following a patch.

The interfaces server has, in general, lower memory demands than the primary Tomcat, but can still run into its memory limits when many webapps are deployed.

When Tomcat runs out of memory customers will most commonly call in with complaints of application slowness as the Java Virtual Machine desperately tries to find enough free memory to meet the needs of all of its threads.

In extreme circumstances Tomcat may even crash. On AIX look under $CATALINA_HOME/bin for files matching the pattern javacore*.txt, containing the current date and time in the filename. When the interfaces server is involved, look for javacore files under its own bin directory. Scan the top of the file looking for a message like this:

0SECTION       TITLE subcomponent dump routine
           ===============================
1TICHARSET     ISO8859-1
1TISIGINFO     Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Failed to allocate JNIEnv" received
1TIDATETIME    Date:                 2015/11/23 at 21:31:07

When you see OutOfMemoryError or something similar that means you can go ahead and turn up Tomcat's memory. Similar messages may appear in the $CATALINA_HOME/logs/catalina.out logfile on Linux and Windows.

Another clue is when a Tomcat's CPU performance spikes and their red "used memory" line approaches the black "max memory" line as seen in the Performance Webapp.

Adjusting Tomcat Memory on Windows

On Windows, the Tomcats run as system services. They are not launched from the .bat files under %CATALINA_HOME%/bin, neither are the .sh files used on Windows.  The Tomcat services are completely configured from the registry, so it is necessary use regedit.exe to adjust its settings.

Recommended: The quick and easy way

• Go to Start Menu -> Run and run regedit
   
• Browse to HKEY_LOCAL_MACHINE --> SOFTWARE --> Wow6432Node --> Apache Software Foundation --> Procrun 2.0 --> TomcatSpillmanServer --> Parameters --> Java
  
   
   
• Edit the JvmMx option (max memory).  There is a radio button that lets you do it in either hexadecimal or decimal. Most people prefer decimal, so you might want to set the radio button to that.
  
   
   
• After exiting regedit, restart tomcat and double check on the Application Manager web page that the setting took effect they way you expect.

The “Official Tomcat Way”, which has proven slightly less reliable

Tomcat comes with a program that will let you adjust the java settings for its service.  Its use is not recommended, but it is explained here for completeness.

• The first thing you need to know is the name of the service Tomcat is running as. Pull up the windows Services dialog, right click on the Tomcat service you want to adjust, and choose Properties.  You will see a page that looks like this:
  
  
  
  The Service name: line tells you the name of the service.  In this case, it’s “Tomcat-SpillmanServer”.
• To run it, go to a command prompt via Run as Administrator (it won’t work in Windows 2008 if you aren’t administrator), change to the tomcat\bin directory, and run:
  tomcatw //ES//Tomcat-SpillmanServer
  *Notice that it’s //ES, as in, “Edit Service”
  *Substitute “Tomcat-SpillmanServer” with the service name you found in the step above
• Running the above tomcat command will pull up a window.  Go to the “Java Args” tab and add or edit the –Xmx option. Usually a good default is –Xmx1024m, which gives tomcat 1GB of RAM (1024 MB).
• Restart tomcat for the setting to take effect.

Adjusting Tomcat's Memory on Linux

Beginning with the 1507 patch the preferred method is to adjust the hook scripts. These scripts offer a more flexible and simple way to manage these settings going foward. They do, however, require manual intervention to migrate the settings from the old scripts the first time. So you should be familiar with both methods.

Editing catalina.sh (the old technique)

• Edit $CATALINA_HOME/bin/catalina.sh.  Go to the line that contains, “JAVA_OPTS” and add or edit the –Xmx option.  The syntax of this flag is -Xmx{[0-9]+}{mg}. The "m" suffix specifies megabytes while the "g" suffix specifies gigabytes.  A good default for the primary Tomcat is –Xmx4096m
  
  
   
• Restart tomcat for the setting to take effect.

Editing the hook script (recommended in 1507 and later)

• Edit $CATALINA_HOME/bin/start.hooks/50-java_opts.hook
• Locate the stanza that governs the -Xmx option and adjust the hardcoded value in both lines. In 1511 this is further simplified by putting all adjustable values in variables at the top of the script
• Locate the MAX_MEMORY= line. This is where you edit the memory value. In the screenshot below, the value is 2048m
  
  
  

Migrating settings from the old catalina.sh script into the new hook script
Because we don't want to throw away customers' current settings, the catalina.sh script isn't automatically changed by our patch tool at this time.  Following a patch or an upgrade these settings must be manually migrated from catalina.sh into the hook script.

The hook scripts are written such that configuration options defined in catalina.sh take precedence over the hook script. These settings must be removed from catalina.sh and put into the corresponding hook script instead. Each setting in JAVA_OPTS is considered individually; this means the mere presence of the JAVA_OPTS variable in catalina.sh doesn't render a hook script completely useless; but that particular setting's value comes from catalina.sh instead of the hook script.

Similarly, the CATALINA_OPTS variable in catalina.sh will override the settings in start.hooks/50-catalina_opts.hook.

Your task is to edit both files and copy settings from catalina.sh into the corresponding hook script. The result should be that catalina.sh does not set any value for JAVA_OPTS or CATALINA_OPTS.