jump to navigation

Apache Tomcat Tuning Guide March 13, 2013

Posted by Terry.Cho in Java Programming, Performance & Capacity Planing.
Tags: ,
3 comments

Tomcat Tuning Guide

 

Tomcat configuration $Tomcat/conf/server.xml

Assumption

This configuration is optimized for REST/HTTP API call. And it doesn’t use any reverse proxy like Apache, NginX etc. We will reside simple L4 switch infront of tomcat groups.

In addition we will not use Tomcat Clustering, Session etc. So the clustering configuration is omitted.

Listener Setting

<Listener className=”org.apache.catalina.security.SecurityListener” checkedOsUsers=”root” /> 

checkedOsUser setting means Unix system user “root” cannot start Tomcat. If user starts tomcat as a root user it makes log file as a root user permission. In that case tomcat user cannot delete the log file.

<Listener className=”org.apache.catalina.core.JreMemoryLeakPreventionListener” /> 

This makes detect memory leak.

Connector Setting

protocol=”org.apache.coyote.http11.Http11Protocol” 

It makes tomcat use BIO. Tomcat has options for IO (BIO,NIO,APR). APR is fastest IO setting. It uses Apache web server IO module, so it is fastest. But it uses C code (JNI call), it can have a risk to kill tomcat instance. (with core dump). APR is more faster about 10% than BIO. But BIO is more stable. Use BIO. (Default is BIO)

acceptCount=”10″

It specifies server request queue length. If message is queued in the request queue, it means server cannot handle incoming message (it is overloaded). It will wait for idle thead and the request message will be pending. This setting reduce total size of request queue to 10. If the queue has been overflowed, client will get a error. It can protect server from high overload and let system manager to know the server has been overloaded.

enableLookups=”false”

In Java Servlet Code, user can look up request message origin (IP or URL). For example user in yahoo.com send request to server, and Tomcat try to resolve incoming request IP address. “enableLooksups” option enables return DNS name not a IP address. During this processing Tomcat look up DNS. It brings performance degradation. This option removes DNS look up stage and increase performance.

compression=”off” 

We are using REST protocol not a normal web contents like HTML,Image etc. This options allows to compress HTTP message. It consumes computing power but it can reduce network payload. In our environment compression is not required. It is better to save computing power. And in some particular Telco network, compression is not supported.

 connectionTimeout=”10000″

It is HTTP Connection time out (client to server). It is milliseconds. (10,000 = 10 sec).

If server cannot make a connection from client til 10 sec. It will throw HTTP time out error. In normal situation, our API response time is under 5 sec. So 10 sec means, server has been overloaded. The reason why I increased the time up to 10 sec is, depends on network condition, connection time will be deferred.

maxConnections=”8192″

The maximum number of connection, tomcat can handle. It means tomcat can handle maximum 8192 socket connection in a time. This value is restricted by Unix system parameter “ulimit –f” (You can check up in unix console)

maxKeepAliveRequests=”1″

As I mentioned above, this configuration is optimized to REST API request not a common web system. It means client will send REST API call only. It sends the request and get a response. Client will not send request in a short time. It means we cannot reuse the connection from the client. So this setting turn of HTTP Keep Alive. (After response the request from client, tomcat disconnect the connection immediately)

maxThreads=”100″

This defines total number of thread in Tomcat. It represents max number of active user at that time. Usually 50~500 is good for performance. And 100~200 is best (it is different depends on use case scenario).

Please test with 100 and 200 values and find value for performance. This parameter also get a impact from DB connection pool setting, even if we have a lot of thread , and the total number of db connection is not enough, the thread will wait to acquire the connection.

tcpNoDelay=”true”

This allows us to use TCP_NO_DELAY in tcp/ip layer. It makes send small packet without delay. In TCP, to reduce small package congestion, it gathers small packet to tcp buffer until it has been filled and send the packet. TCP_NO_DELAY option makes send small packet immediately even though TCP buffer is not full.

 

JVM Tuning

Java Virtual Machine tuning is also very important factor to run Tomcat

The focus of JVM tuning is reducing Full GC time.

-server

This option makes JVM to optimize server application. It tunes HotSpot compiler etc internally. This option is very important and mandatory in server side application

-Xmx1024m –Xms1024m -XX:MaxNewSize=384m -XX:MaxPermSize=128m

This memory tuning options, our infrastructure is using c1.mediuem amazon instance, so the available memory is about 1.7 gb total. Heap size is 1G and let them to have fixed size. It defines max 1Gb, min 1Gb heap size. The NewSize is 384mb (1/3 size of total heap size). 1/3 New Size is best performance usually. Perm size is defines area of memory to load class. 64mb is enough. But we will use 128m first time and tune based on gc log analysis later.

Total physical memory consumption is 1G heap + 128mb perm = 1.128 GB and JVM internally uses memory to run JVM itself. It consumes about 350~500mb. So total estimated required memory is about 1.128GB+500m = 1.5 GB.

As I mentioned, c1.mediuem size has only 1.7GB physical memory. If consumed memory exceeds actual physical memory, it makes disk swapping. If JVM memory is swapped out to disk, the performance is significantly degraded. Please take care swapping is not occurred.

-XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./java_pid<pid>.hprof

These options are for trouble shooting “OOM (Java Out Of Memory Error”. If out of memory error has been occurred. The memory layout will be dumped to disk. The location of dumpfile is specified by “-XX:HeapDumpPath” option

 -XX:ParallelGCThreads=2 -XX:-UseConcMarkSweepGC

These options specify GC strategy. It uses ParallelGC for Minor collection and 2 threads will be used for the Minor GC. And for Old area, concurrent gc will be used. It will reduce Full gc time

-XX:-PrintGC -XX:-PrintGCDetails -XX:-PrintGCTimeStamps -XX:-TraceClassUnloading -XX:-TraceClassLoading

These option specifies GC logging. It logs the GC log detail to stderr (console output). It shows usage trend os Java Heap memory, time stamp etc. (it contains old,new & perm area usage).

Especially, ClassLoading & UnLoading option show what class is loaded and unloaded to memory. It helps us to trace Perm Out of memory error.

 

Reference : http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

Logging

  1. 모든 log는 catalina.out 하나의 파일에 모두 쌓이게 할것 à LogAppender를 Console Appender로 변경하면 된다.

①     Tomcat 자체가 쓰는 로그 (별도로 지정 안하면 원래 Console로 나옴)

②     Application에서 LogBack을 이용해서 로깅 되는 로그 (별도로 ConsoleAppender를 개발단에서 정의해줘야 함)

③     GC 로그 (별도로 지정 안하면 원래 Console로 나옴)

DB Connection Pool Setting

Please use tomcat dbpcp connection pool. Apache-common connection pool is not updated frequently. Tomcat dbcp connection pool is updated well.

The basic setting guide is “Let number of connection in the pool to keep exact number of connection”. It can be done by set min conn and max conn to same number.

Total number of connections (including read + write) should be around. 70~80. We are using 100 threads in one tomcat instance. The 70~80% will use db connection at the same time.

 

Please attach. Current connection pool settings here

 

Library Setting

Developer packaged java lib inside war file (WEB-INF/lib) it can increase usage of perm memory. And sometime can bring confusion about “which lib is actually used”. It means if same lib(jar file) resides in $TOMCAT_HOME/lib and WEB-INF/lib. The lib in $TOMCAT_HOME/lib wil be used and lib is WEB-INF/lib will be ignored.

To solve this problem. Remove common library like my-sql-jdbc driver and dbcp lib jar file from WEB-INF/lib and move it into $TOMCAT_HOME/lib

Setup ADF Faces based JSF Runtime environment in Tomcat 6 July 24, 2009

Posted by Terry.Cho in JSF & ADF (Oracle Application Development Framework).
Tags: , , , ,
6 comments

Tutorial for set up ADF Faces for Tomcat 6.0.
I used JDeveloper 11 version.

If you feel the steps are hard, you can easily download below files and exract them into $TOMCAT_HOME/lib

http://www.box.net/shared/4uppskee7z

Important :
If  you are using JDeveloper to develop ADF Faces application. Please uncheck all of Libraries in WEB-INF/lib in your application. The libraries are already deployed by above steps. If you deploy them by including WEB-INF/lib. It can make error.

Deploy ADF based JSF application into WebLogic Server July 23, 2009

Posted by Terry.Cho in JSF & ADF (Oracle Application Development Framework).
add a comment

This posting is continued from https://javamaster.wordpress.com/2009/07/23/develop-jsf-application-with-oracle-adf-in-eclipse/
It explains how to configure weblogic domain and deploy ADF based JSF application to weblogic server. And test the application.

Develop JSF application with Oracle ADF in Eclipse. July 23, 2009

Posted by Terry.Cho in JSF & ADF (Oracle Application Development Framework).
Tags: , , , ,
11 comments

Oracle ADF has JSF (Java Server Faces – Part of JEE5 standard) implementation.
It provides very rich UI component.
Commonly people think that “to use ADF, do we need JDeveloper?”. Answer is no.
ADF contains whole application framework including Biz Logic, Data Access,View etc.
In this example we just use ADF Faces as a JSF implementation and shows implementation method with eclipse.

Environment
– WebLogic 11g (Bundled in JDeveloper 11g)
– JDeveloper 11g
– Eclipse Galileo

You can download source code here