activiti engine hangs every few hours in tomcat

[plehal]

I am using Activti in tomcat. Everything works as intended but I have a very intriguing problem. The Activiti engine freezes after 1-2 hours and nothing happens after that until I restart the tomcat. There are no exceptions thrown..no error log etc. It just stops working. It is not the memory usage in tomcat either as all other applications keep working, only the applications which use Activiti engine stop working. Can someone throw a few pointers where I should start looking in the absence of any clues…………?

[ronald_van_kuij]

Look if there are free database connections…

[plehal]

Thanks for the clue. I will check it when the server freezes next time. Unfortunately it is arbitrary when it happens and can not be reproduced consistently. I tried stress testing using Jmeter but could not reproduce the freeze condition.

[trademak]

You see nothing in the log at all?
Are you using Activiti embedded, or are you using the Activiti REST app?
Every call to Activiti hangs?

Best regards,

[plehal]

I am using Activiti as WFE library in my WebApp. Freeze happens only randomly , without any log. It may be due to database connection unavailability as previously mentioned. However, in continuous stress testing it did not freeze……. may be because connections are alive due to faster requests.  I am using this in production environment (it was a challenge as there was a need for a solution in place within 4 weeks and none of the commercial vendors were  able to meet the requirements- technical, budget and timing). Anyway, I am restarting tomcat servers in the cluster every hour so the end users don't even know about this….   I have jdbcMaxActiveConnections at 50 and in dev and QA freezing has not occurred since I bumped up this number but I want to be sure if that is what caused the freeze in the first place.

[plehal]

Db connections does not seem to be the issue as far as I can see from MySQL end. However, if the activiti dbpool is somehow failing to get live connections – for any reason , I am at a loss to find that…. right now I am seeing only 3 connections open in MySQL database but activiti is frozen. More clues please!!!!!!!!!!!

[ronald_van_kuij]

Debug the connection pool…

[jbarrez]

Which Activit version? Old versions of Activiti did indeed have a connection pool bug, but that's fixed.
Which connection pool are you using? It sure makes a difference, see http://www.jorambarrez.be/blog/2012/04/30/dbcp_vs_c3p0_bonecp/

[plehal]

The problem existed with 5.8, 5.9 and now with  5.10, however, the issue seems to be exactly the same as described in the article.

[jbarrez]

yes, that's what I'm suspecting. The default ibatis connection pool isn't tuned for really heavy load I'm afraid.
Give bonecp a go and let us know how it works out for you!