topic Re: Clustering in Alfresco Archive

Clustering

derikalov — Wed, 20 Feb 2013 04:36:04 GMT

I know that Activiti is "just a jar" but what happens if there are N instances (JVMs) running against the same Activiti db?What if I want to achieve some load-balancing and fault tolerance?Is there an easy way to achieve that? I think that everything except timerEvent-triggered processes can be clea

Re: Clustering

jbarrez — Wed, 20 Feb 2013 12:24:09 GMT

I know that Activiti is "just a jar" but what happens if there are N instances (JVMs) running against the same Activiti db?

Not much, than all those engines will operate on the same data

What if I want to achieve some load-balancing and fault tolerance?

Then you'll need typical solution which are used to make Java webapps highly available: tomcat cluster, loadbalancer in front, etc. Nothing fancy there. The activiti API is stateless and any node can take over from each node.

Is there an easy way to achieve that? I think that everything except timerEvent-triggered processes can be cleanly split across N engines. But for timerEvent triggered jobs, can I have some hooks into the engine so it first does some kind of election of which engine picks up the process - and the other engines then ignore it.

For timers you can even dedicate one specifc machine which only runs the job executor. (or more than one to make them fault tolerant). Both the job executor and Activiti in general is designed in a way that works clusterable out of the box.

Re: Clustering

derikalov — Wed, 20 Feb 2013 21:13:40 GMT

Thank Joram, but my question is HOW do I dedicate? At this point I don't know enough about Activiti internals. My impression is that as long as 2 instances share the same database, they will both pick up and run the timer job. But how do I make sure that only 1 of them picks up a job? And if that engine stops working how do I get the "standby" engine to start picking up timer jobs?

Re: Clustering

jbarrez — Thu, 21 Feb 2013 09:27:11 GMT

But how do I make sure that only 1 of them picks up a job?

You don't. The job executor is written in such a way that it is clusterable. Only one job executor will pick up the jobs. Internally, this is done by first acquiring the job (locking it for a certain time) before it is actually executed. The locking happens using optimistic locking making sure no two jobs are picked up by different threads.

So I meant it when I said Activiti is written with cluserability in mind 😉

And if that engine stops working how do I get the "standby" engine to start picking up timer jobs?

That will also happen automatically, as it will just keep picking and executing timers as before, but now the chance on optimistic locks is of course much smaller (or not existant).

Re: Clustering

derikalov — Mon, 25 Feb 2013 20:11:14 GMT

Hi Joram, this is great! This is great information, I wish it was more prominently stated in the User Guide or a separate doc, perhaps, Admin guide?

One more question, how do "dedicate one specifc machine which only runs the job executor"? Could not find anything specific in the docs…

Re: Clustering

jbarrez — Tue, 26 Feb 2013 10:01:50 GMT

Hi Joram, this is great! This is great information, I wish it was more prominently stated in the User Guide or a separate doc, perhaps, Admin guide?

That is certainly true. But to do that, we'd like to set up a clustered 'reference' architecture up first before writing it down. But point taken, indeed.

One more question, how do "dedicate one specifc machine which only runs the job executor"? Could not find anything specific in the docs…

That is also pretty easy: simply have one activiti.cfg.xml with the job executor set to enabled, and all the rest to disabled. If you want two machines doing the jobs, simply enable it on the other machine.

<property name="jobExecutorActivate" value="false" />

Re: Clustering

schitale — Tue, 06 May 2014 18:59:29 GMT

We have been having quite a lot of issues with clustering.
We have two servers in a cluster. On both of them we have job executor running. Very often we see optimistic locking exception and very often we see variable values that we are storing are getting lost. This typically happens during the same time.

I also see the execution happening on different servers at the same time.

Here is the scenario:

–>(Java Delegate)–>(Async Task)–>

These async tasks and java tasks are exclusive. But I guess that exclusive behavior is only limited in one server.
(http://bpmn20inaction.blogspot.com/2012/12/implement-parallel-execution-in-activiti.html)

In java delegate we do setVariable() and very often we see values being lost after Async task. Is this how it is supposed to work? Does the Async Taks gets executed before the values are stored by (java Delegate) ?