parallel instantiation of workflow depending on the DB count

[nirvana]

Hi All,

Attached the workflow.

I'm able to instantiate the workflows parallelly depending in regards with some X count from DB.

If the count is more than 22 then am getting stack overflow error.
In my case, I have a requirement of handling more than 50000 in one go….

[nirvana]

stack trace for the above issue

16:58:00,119 INFO  [stdout] (http–127.0.0.1-8080-1) in PlanningPhase phase Count ==79
16:58:00,134 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningPhase execute() : END
16:58:00,134 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningCycleProcess execute() : START
16:58:00,150 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleProcess updatePlanningCycleWFDetails() : START
16:58:00,150 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : updatePlanningCycle() START
16:58:00,165 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : updatePlanningCycle() END
16:58:00,165 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleProcess updatePlanningCycleWFDetails() : END
16:58:00,165 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : getPlanningCycleVO() START
16:58:00,165 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : getPlanningCycleVO() END
16:58:00,165 INFO  [stdout] (http–127.0.0.1-8080-1) PLAANNING PHASE COUNT====79
16:58:00,165 INFO  [stdout] (http–127.0.0.1-8080-1) %%%%%%%%%%%% planningPhaseID is : 22
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) %%%%%%%%%%%% planningPhaseID is : 22
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) execution==ConcurrentExecution[785689]
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningCycleProcess execute() : END
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningPhase execute() : START
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) <<< ======= Phase ID in PlanningPhase is =======>>> 22
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningPhaseProcess updatePlanningPhaseWFDetails() : START
16:58:00,181 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : updatePlanningPhase() START
16:58:00,212 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : updatePlanningPhase() END
16:58:00,212 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningPhaseProcess updatePlanningPhaseWFDetails() : END
16:58:00,212 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : getPlanningCycleVO() START
16:58:00,212 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : getPlanningCycleVO() END
16:58:00,212 INFO  [stdout] (http–127.0.0.1-8080-1) in PlanningPhase phase Count ==78
16:58:00,212 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningPhase execute() : END
16:58:00,228 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningCycleProcess execute() : START
16:58:00,228 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleProcess updatePlanningCycleWFDetails() : START
16:58:00,228 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : updatePlanningCycle() START
16:58:00,244 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : updatePlanningCycle() END
16:58:00,244 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleProcess updatePlanningCycleWFDetails() : END
16:58:00,244 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : getPlanningCycleVO() START
16:58:00,244 INFO  [stdout] (http–127.0.0.1-8080-1) In PlanningCycleDAO : getPlanningCycleVO() END
16:58:00,244 INFO  [stdout] (http–127.0.0.1-8080-1) PLAANNING PHASE COUNT====78
16:58:00,244 INFO  [stdout] (http–127.0.0.1-8080-1) %%%%%%%%%%%% planningPhaseID is : 23
16:58:00,259 INFO  [stdout] (http–127.0.0.1-8080-1) %%%%%%%%%%%% planningPhaseID is : 23
16:58:00,259 INFO  [stdout] (http–127.0.0.1-8080-1) execution==ConcurrentExecution[785703]
16:58:00,259 INFO  [stdout] (http–127.0.0.1-8080-1) ++++++++++++++++++++++++++++++++++++++++In PlanningCycleProcess execute() : END
16:58:02,665 SEVERE [com.vaadin.Application] (http–127.0.0.1-8080-1) Terminal error:: com.vaadin.event.ListenerMethod$MethodException: Invocation of method buttonClick in org.activiti.e
xplorer.ui.task.TaskDetailPanel$5 failed.
        at com.vaadin.event.ListenerMethod.receiveEvent(ListenerMethod.java:530) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.event.EventRouter.fireEvent(EventRouter.java:164) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.ui.AbstractComponent.fireEvent(AbstractComponent.java:1219) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.ui.Button.fireClick(Button.java:567) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.ui.Button.changeVariables(Button.java:223) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.changeVariables(AbstractCommunicationManager.java:1460) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleVariableBurst(AbstractCommunicationManager.java:1404) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleVariables(AbstractCommunicationManager.java:1329) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.doHandleUidlRequest(AbstractCommunicationManager.java:761) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.terminal.gwt.server.CommunicationManager.handleUidlRequest(CommunicationManager.java:325) [vaadin-6.8.8.jar:6.8.8]
        at com.vaadin.terminal.gwt.server.AbstractApplicationServlet.service(AbstractApplicationServlet.java:501) [vaadin-6.8.8.jar:6.8.8]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
        at org.activiti.explorer.filter.ExplorerFilter.doFilter(ExplorerFilter.java:44) [activiti-explorer-5.13.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
        at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
Caused by: java.lang.StackOverflowError
        at java.util.HashMap.put(HashMap.java:372) [rt.jar:1.6.0_30]
        at org.activiti.engine.impl.db.DbSqlSession.cachePut(DbSqlSession.java:340) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.db.DbSqlSession.insert(DbSqlSession.java:116) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.history.HistoryManager.recordActivityStart(HistoryManager.java:178) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.history.handler.ActivityInstanceStartHandler.notify(ActivityInstanceStartHandler.java:30) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.pvm.runtime.AbstractEventAtomicOperation.execute(AbstractEventAtomicOperation.java:42) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:87) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:532) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:527) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionCreateScope.execute(AtomicOperationTransitionCreateScope.java:49) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:87) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:532) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:527) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerTake.execute(AtomicOperationTransitionNotifyListenerTake.java:66) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:87) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:532) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:527) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionDestroyScope.execute(AtomicOperationTransitionDestroyScope.java:116) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:87) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:532) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:527) [activiti-engine-5.13.jar:5.13]
        at org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerEnd.eventNotificationsCompleted(AtomicOperationTransitionNotifyListenerEnd.java:36) [activiti-engin

PLease suggest for getting this out.

thanks in advance.

regards,
Nirvana

[frederikherema1]

This is a known issue in the command-processing inside the engine. We have an issue open for this for a while now, but fail to make the tail-recursion prevention work in all cases.

A solution whould be to make use of asynchronous activities. I see you're always looping back to the same call-activity to start a process. Perhaps, also take a look at the Multi-instance capabilities. This prevent having a deep sequence-flow stack, since the loop is handled by the multi-instance behaviour rather than depending on process-flow.

[nirvana]

Thanks Frederik Heremans.

As suggested I'm using Multi-Instance modelling, I'm able to see 10K instances in 5 mins with 8GB memory and 16M stacksize . But when I configure the count to 100K literally I see 250 instances are created and after that I see OutOfMemory….

Any suggestions plz…


Regards,
Nirvana

[frederikherema1]

Are they executed parallel, or sequential (the multi-instance activity, I'm referring to)? Any reason why you would try to automate 100K loops? Why not split it up in multiple processes?

[nirvana]

My requirement - as per the diagram above, when I start and complete the first step(planning cycle task) then all the related planning phases should be started and initialized.

Here can you please clarify me what exactly do you mean 'Why not split it up in multiple processes' ?

Regards,
Nirvana

[nirvana]

Any help or suggestions on the above ?

Regards,
Nirvana

[nirvana]

Hi,

Also here I have question like in the above multi process instance model when I use Sub process I'm getting same ID for all the 'n' tasks.

But when I use call activiti instead of sub-process I'm getting all the different id's correctly.

Can you please tell me what is the difference between these ? everything else is same ? just changed the sub-process to call activiti everything is working properly.

Watinig for the response.
Thanks in advance.

Regards,
Nirvana

[jbarrez]

Another solution is to split up processes, and use async on certain steps.

What 'id' do you mean? Database id or activity id?

[nirvana]

HI All,

Thanks for the response.
Its a database ID.
tried splitting up the process, but still there no luck ?

Is this the only way ?

Regards,
Nirvana