Running multiple instances of same process (with Camel routes)

[ghollins]

Hi,

   I'm having some difficulty running multiple instances of the same process at the same time.  I'm kicking them off on separate Java threads as suggested in other forum topics.  However, they seem to be running one after the other in a serial fashion, instead of in parallel.    

My process flow is almost exactly the same as diagram with receive tasks shown here:

http://bpmn20inaction.blogspot.com/2012/12/implement-parallel-execution-in-activiti.html

As a clarification, multiple instances of the same processes seem to be successfully going in parallel, until the receive tasks hit.  In other words I see a progression like:


03:26:41,857 [pool-19-thread-1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ProcessInstance[133] takes transition (startevent1)–sequenceflow1–>(parallelgateway1)
03:26:41,857 [pool-19-thread-2] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ProcessInstance[134] takes transition (startevent1)–sequenceflow1–>(parallelgateway1)
…
03:26:41,857 [pool-19-thread-2] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - transitions to take concurrent: [(parallelgateway1)–flow1–>(servicetask1), (parallelgateway1)–flow3–>(servicetask2)]
03:26:41,858 [pool-19-thread-1] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - transitions to take concurrent: [(parallelgateway1)–flow1–>(servicetask1), (parallelgateway1)–flow3–>(servicetask2)]
…
…
03:26:41,858 [pool-19-thread-2] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - new ScopeExecution[139] with parent ProcessInstance[134] created to take transition (parallelgateway1)–flow1–>(servicetask1)
03:26:41,858 [pool-19-thread-1] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - new ScopeExecution[140] with parent ProcessInstance[133] created to take transition (parallelgateway1)–flow1–>(servicetask1)
…
…
03:26:41,858 [pool-19-thread-2] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - new ScopeExecution[141] with parent ProcessInstance[134] created to take transition (parallelgateway1)–flow3–>(servicetask2)
03:26:41,858 [pool-19-thread-1] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - new ScopeExecution[142] with parent ProcessInstance[133] created to take transition (parallelgateway1)–flow3–>(servicetask2)
…
…

03:26:41,860 [pool-19-thread-1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[140] takes transition (parallelgateway1)–flow1–>(servicetask1)
03:26:41,860 [pool-19-thread-2] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[139] takes transition (parallelgateway1)–flow1–>(servicetask1)
…
…
03:26:41,863 [pool-19-thread-1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[140] takes transition (servicetask1)–flow8–>(receivetask1)
03:26:41,864 [pool-19-thread-2] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[139] takes transition (servicetask1)–flow8–>(receivetask1)
…
…
03:26:41,866 [pool-19-thread-1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[142] takes transition (parallelgateway1)–flow3–>(servicetask2)
03:26:41,867 [pool-19-thread-2] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[141] takes transition (parallelgateway1)–flow3–>(servicetask2)
…
…
03:26:41,869 [pool-19-thread-1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[142] takes transition (servicetask2)–flow4–>(receivetask2)
03:26:41,870 [pool-19-thread-2] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[141] takes transition (servicetask2)–flow4–>(receivetask2)
…
…
03:26:51,901 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[140] takes transition (receivetask1)–flow9–>(parallelgateway2)
…
…
03:26:51,931 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - transitions to take concurrent: [(parallelgateway2)–flow7–>(endevent1)]
…
…
03:26:51,939 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ProcessInstance[133] takes transition (parallelgateway2)–flow7–>(endevent1)
…
… =================================
… NOTE LONG GAP IN TIME HERE (ABOUT 10 SECONDS)
… =================================
…
03:27:01,922 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[139] takes transition (receivetask1)–flow9–>(parallelgateway2)
…
…
03:27:01,948 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ConcurrentExecution[141] takes transition (receivetask2)–flow6–>(parallelgateway2)
…
…
03:27:01,953 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.persistence.entity.ExecutionEntity  - transitions to take concurrent: [(parallelgateway2)–flow7–>(endevent1)]
…
…
03:27:01,960 [Camel (camelContext) thread #7 - seda://receiveQueue1] DEBUG org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake  - ProcessInstance[134] takes transition (parallelgateway2)–flow7–>(endevent1)


+++++++++++++++++++++++++++++++

So there is about a 10 second delay.  This looks to me like if one route is tied up in camel, then the other one can't take it until the first route returns.  I'm basically doing a 10 second sleep in the bean that does the work (to simulate real work), before returning… 

Any ideas how to work around this?

My camel routes look like:

++++++++++++++++++++++++++++++++++++++++++++++++++++++

from("activitiarallelProcessTest:servicetask1?copyVariablesToProperties=true").setHeader("destination", constant("activitiarallelProcessTest:receivetask1")).to("seda:asyncQueue1a");

from("seda:asyncQueue1a").to("bean:longTask?method=runLongTask(10000)").to("seda:receiveQueue1");
      
from("activitiarallelProcessTest:servicetask2?copyVariablesToProperties=true").setHeader("destination", constant("activitiarallelProcessTest:receivetask2")).to("seda:asyncQueue2a");

from("seda:asyncQueue2a").to("bean:longTask?method=runLongTask(10000)").to("seda:receiveQueue1");
      
from("seda:receiveQueue1").recipientList(header("destination"));

+++++++++++++++++++++++++++++++++++++++++++++++++++++++


Thanks!

[ghollins]

I think I figured this out myself after further investigation.  It turns out that the Camel route by default has a 'concurrentConsumers' setting of 1 (see http://camel.apache.org/jms.html).  By bumping this up to a higher number (>= the number of processes I wanted to run concurrently), I was able to overcome this behavior.

[frederikherema1]

Great, thanks for sharing the solution!