Activiti on Websphere and Spring AsyncJobExcutor

[ggr]

Context : Make long story short, it has been decided in my company to use Activiti, we have strong knowledge in EMC Workflow Engine. We have deployed it in a hurry in production on IBM Websphere Application Server.
Not enough load/perfomance tests, the inevitable happens : the server crashed.

After some investigations, blog posts & forums readings we found a solutions.

[h2]1. Work Manager et Timer Scheduler[/h2]

Managing your own thread on a WAS is never a good idea. Even if the new class ManagedAsyncJobExecutor use a threads pool of the container this is not suitable for the WAS.
It use work manager - a pool of threads - and managed it by itself.

I re-wrote the existing DefaultAsyncJobExecutor using the spring class org.springframework.core.task.AsyncTaskExecutor, it works like the java java.util.concurrent.ExecutorService.
This interface is mainly implemented by two classes

• one for a Servlet Container : org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor
• one for a Applicaton Container : org.springframework.scheduling.commonj.WorkManagerTaskExecutor (needs the jndi name of the work manager)

   
The AbstractAsyncJobExecutor starts two threads two retrieve jobs to treat. Regarding the code, this is just a runnable snippet that is scheduled.

I rewrite the two classes AcquireAsyncJobsDueRunnable & AcquireTimerJobsRunnable, the common code (99%) has been mutualized in AbstractAcquireJobsRunnable.
I use the time scheduler to avoid Thread.sleep(). I use the Spring implementation because of the WAS, but a standard java one does exist java.util.concurrent.ScheduledExecutorService.
I think this is better to use this than a thread.

The philosophy is still the same :

• The Async executor starts schedule of execution of the AcquireAsyncJobsDueRunnable & AcquireTimerJobsRunnable
• When the scheduler is executed, two tasks are executed by the AsyncTaskExecutor - one for each runnable
• They check the database to retrieve jobs and put runnable task for each job in the AsyncTaskExecutor.
• if there is no more job to treat or if the queue of the AsyncTaskExecutor is full, the AbstractAcquireJobsRunnable schedule its next execution.

I'm a little bit annoyed with the term AsyncJobExecutor because it does more than just executing job. It rerieves jobs to treat and then only executes them.
It is more a AsyncJobExecutorManager or something like that.

One more strange thing is that AcquireAsyncJobsDueCmd constructor requires a AsyncJobExecutor object. In fact, it only needs lockTimeInMillis and maxJobsPerAcquistions params.

Globally, the rewriting of the classes is more OOP / Spring compliant.

[h2]2. Transaction Manager[/h2]

We use the inappropriate transactionManager : org.springframework.jdbc.datasource.DataSourceTransactionManager.
For WAS, you have to use : org.springframework.transaction.jta.WebSphereUowTransactionManager


[h2]3. Websphere Application Server Config[/h2]

Define a workmanager :

• name : workmanager/listener
• Work request queue size　: 0
• Work request queue full action　: FAIL
• Alarm threads : 15
• Minimum threads : 10
• Max threads : 200
• Threads priority : 5

Define a scheduler :

• name : timer/listener

[h2]4. Activiti 6 and message queue based async executor[/h2]

The mechanism explained below is applicable for Activiti 6 OOTB.

Following this post http://www.jorambarrez.be/blog/2016/07/20/message-queue-async-executor-benchmark/, we can adapt the mechanism like this based on the producer / consumer pattern.

• Use a scheduler to schedule tasks to retrieve jobs
• job are put in a queue (java, jms, whatever.)
• use a second scheduler to read this queue and put the job in the workmanager

I dont know if it will be more efficient, but it can regulate the load of the executor

Feel Free to ask more details of the solutions.

Thanks for providing such a great product.

Code in attachment, rename it to code.zip

[gdharley]

Hello GGR, this is fantastic work that I expect will be very useful for other community members.
Would Fujitsu be prepared to contribute this as an alternative Job Scheduler?

Thanks,
Greg

‌