Friday, 28 October 2011

Glassfish 3.1 cluster: Multiple client thread accessing same session

 I faced a problem while deploying a web application in Glassfish 3.1 cluster. The problem was that HTTPsessions were getting lost haphazardly. I was trying all possible sorts of configurations but the problem was not getting away. While testing the application, I observed that mostly at the time of loading the menu/events of the application the session was getting lost. After giving some load on Google servers, I found the cause of the problem. As per the Glassfish High Availability Administration Guide, if we use Memory Replication in our Web Application and if multiple client threads concurrently access the same session ID, then we may experience session loss without any instance failure. Glassfish 3.1 memory replication framework makes use of session versioning. As our application was using multiple threads concurrently to access the same session ID (mainly at the time of loading menu/events of the page, or on some ajax calls), so the session was getting lost. The solution for this problem was to add <property name="relaxCacheVersionSemantics" value="true"/> in the glassfish-web.xml file.

    <session-manager persistence-type="replicated">
                                <property name="relaxCacheVersionSemantics" value="true"/>
                                <property name="persistenceFrequency" value="web-method"/>

This enables the web container to return for each requesting thread whatever version of the session that is in the active cache regardless of the version number. And voila, I got it right and it worked.   

Friday, 21 October 2011

Using Glassfish Monitoring and finding JDBC connection leak

Recently one of our applications was facing one problem. Problem was that it stops responding after running for some time mainly during load period. I checked logs but could not find any clue about the problem. After some observation I found that static pages were opening normally and the problem was coming only in dynamic pages. So I was sure that the problem is with the database connectivity and checked the JDBC connection pool settings. But connection pool settings were OK, and then I decided to enable monitoring of Glassfish. To enable monitoring go to
Configurations -> server-config -> Monitoring

Checked the Enabled checkbox next to the Monitoring Service
There are three Monitoring Levels (OFF, LOW, HIGH) defined for each individual component. Right now my main concern is about the JDBC Connection Pool, so I will make monitoring level HIGH against it and save the settings.

To check the monitored data go to the Monitor tab in server link

In the Monitor tab, the monitored data is grouped in different categories like Application, Server and Resources depending on the monitored data type. As I want the JDBC Connection Pool data so I have to go to the Resources tab.

When I checked the statistics for the connection pool, I found that a value of more than 100 for WaitQueueLength. This means that much of connection requests were in the queue. The maximum number of connections for that pool were already used, so the new connection requests were going to the queue. But as per my experience, the maximum number of connections for that pool was quite sufficient according to the load on the application. So I suspected connection leak, and I decided to enable connection leak monitoring for this connection pool.

To enable connection leak monitoring, go to the Advanced tab of the connection pool

To enable connection leak monitoring, enter a non-zero value for Leak Timeout. Default is 0 which means leak monitoring is disabled.
If we enable Leak Reclaim for the connection pool, then the leaked connections will be reclaimed by the pool after the connection leak timeout occurs.

After enabling connection leak monitoring, when I checked the monitored data after some time, I could see NumPotentialConnectionLeak had a value of 16. This means that some connection leak is occurring.

To find the details of the connection leaks, we can check the server log.

In the Log Viewer, go to Advanced Search

In Log Level select WARNING and in Logger list select javax.enterprise.resource.resourceadapter and search

In the Log Viewer we can find information about the connection leaks

If we click on details link attached to each Message, we can see the details of each message

জয় আই অসম,
প্রণব শর্মা