topic Re: Looping Activiti Workflow in Alfresco Archive

Looping Activiti Workflow

coner — Mon, 25 Jul 2011 18:41:06 GMT

I have been trying to implement a daemon process that will wakeup and do some work every X time in activiti.I was able to do this with 2 service tasks and an exclusive gate that loops back to the first task, but after the flow executes 92 times it crashes, always.It reports back a stackOverflow erro

Re: Looping Activiti Workflow

ronald_van_kuij — Mon, 25 Jul 2011 18:54:31 GMT

Wrong forum, but I'll respond anyway (next time use the userforum please) because it is a fairly common misconception.

A sleep in a task makes sure you stay in the same thread. So it is indeed recursive. Normally you'll have a real waitstate and e.g. trigger it from an external event. So either use e.g quartz or model timeouts on tasks so they become real waitstates and yo do not get recursive behaviour

Re: Looping Activiti Workflow

coner — Mon, 25 Jul 2011 19:05:12 GMT

shouldn't the workflow processing be sequential though and not recursive? does this mean that i cannot create linear workflows that are larger than 92 steps?

Re: Looping Activiti Workflow

jbarrez — Tue, 26 Jul 2011 08:12:11 GMT

The easiest way to contribute is create a JIRA and add a patch file.

Note that looping support is not an easy task, and should be tested thorougly with plenty unit tests.

Re: Looping Activiti Workflow

ronald_van_kuij — Tue, 26 Jul 2011 15:55:09 GMT

shouldn't the workflow processing be sequential though and not recursive? does this mean that i cannot create linear workflows that are larger than 92 steps?

What you do is not really recursive, it is a loop, a special kind of sequence. There needs to be a kind of 'callstack' to be able to undo things if an exception occurs, or to commit things when things go wrong. The concept of BPM is (most of the time) that there are some kinds of wait states like usertasks, receive tasks etc… If those are modeled in, the 'callstack' is persisted and stack overflows do not occur.

This does indeed mean that e.g. in your case, a lot of servicestasks after another without a waitstate will lead to stackoverflows, whether caused by loops or real sequences does not matter. Adding additional memory helps, but modelling things differently often does as well.

Re: Looping Activiti Workflow

coner — Tue, 26 Jul 2011 17:44:22 GMT

so how would i go about adding more memory to a single workflow that is going to execute?

Re: Looping Activiti Workflow

ronald_van_kuij — Tue, 26 Jul 2011 19:15:32 GMT

Sigh, sorry, you focussed on THAT remark… I'm less willing to help now, in combination with you trying and trying (in PM) to justify wrongly posting in this forum. If you have any IQ, you would not have taken that TO seriously and looked at the second remark… MODDELING THINGS DIFFERENTLY. But for me to help you with that, you should describe what you want to achieve, not focus on a single problem that happens to (almost) noone but you…

Re: Looping Activiti Workflow

coner — Tue, 26 Jul 2011 22:49:12 GMT

obviously you intended to tell me to model things differently…. and obviously the reason that I asked how to change the memory is because I cannot accept the limited amount of memory space that is being allocated to individual workflows AND because runtime execution stack dumps are not acceptable when the workflow may not actually contain recursion or infinite loops.

I would like to be able to change the memory size to model the expected stack space consumption of an entire workflow execution. based on this I can then report back that a designed workflow is most likely going to bust the activiti stack and prevent their deployment before they are instantiated.

Re: Looping Activiti Workflow

filipearaujo — Wed, 27 Jul 2011 14:16:22 GMT

I've exactly the same problem, i want to make a loop with a condition that it may take long or short time, and it gives me the "StackOverflow" exception. I have trying doing this with a exclusive gateway and a parallel gateway and in both cases gives the "StackOverflow" exception. Is there a way to make a efficient loop?

Something like this: http://www.workflowpatterns.com/patterns/control/new/wcp21.php

Re: Looping Activiti Workflow

coner — Wed, 27 Jul 2011 15:28:17 GMT

based on the past few days discussion and as far as i know

short version:
no.

long version:
this is because memory is being added to the stack at every step inside of your workflow (by the engine), never being cleaned up, it will eventually cause a stack overflow.

ronald.van.kuijk's response was to model it differently….

Re: Looping Activiti Workflow

filipearaujo — Wed, 27 Jul 2011 15:45:37 GMT

Yes I know the cause but model it differently? how? Can anybody explain me?

Re: Looping Activiti Workflow

coner — Wed, 27 Jul 2011 16:32:32 GMT

they would prefer that you use an external system that will periodically execute the workflow. this would avoid the memory issue.

i have been able to do it by creating 2 workflows
start -> do looping work -> call sleep -> end
start -> sleep -> call loop -> end

**the calling sleep and calling loop is done via a different communication channel so that the memory dependency can be cleared by the "end" step being reached. essentially the "call sleep" is async.

**there is added overhead in starting and stopping all of these workflows (if the loop is that fast)

**the "sleep" step does not actually have to sleep, the whole point is that the first workflow needs to be able to reach the end state so that memory could be freed.

Re: Looping Activiti Workflow

filipearaujo — Wed, 27 Jul 2011 17:05:59 GMT

It's a little bit awkward but thanks for the tip.

Re: Looping Activiti Workflow

filipearaujo — Thu, 28 Jul 2011 15:42:38 GMT

I don't understand one thing, what do you mean with external system? For instance, I'm doing tests with eclipse plugin with the example that you mention and I created 2 workflows and to call each one other I use CallActivity task but also gives "StackOverflow" exception and with reason because callactivity its synchronous. How can i make an asynchronous comunication?

Re: Looping Activiti Workflow

coner — Thu, 28 Jul 2011 16:16:30 GMT

by external system i mean using JMS / socket / carrier pidgeon
(i am using JMS…. it required an additional controller to actually spawn the things upon death instead of using callactivities)

it needs to be an async call that is not in any way connected to activiti, this will allow the workflow to actually die

more detail:
start -> loop -> JMS start sleep -> end
start -> sleep -> JMS start loop -> end
export both to runnable jar

to allow this to work there is also an activiti_controller (Normal Java Process) that is capable of spawning (via java process exec java -jar workflowName) individual workflows. this controller listens on the JMS queue for messages via a while(true) recvMessage.

this is async because the controller will recv the messages and process them when it is capable, aka re-start the workflows.

(for it to be more robust the controller should be multi-threaded)

this is not the only way to get it to happen, it is just the way that I did it. (yes it is an awful kludge)

let me know if you have any other troubles, i have code and such that works with this via JMS (but i have more components and threading going on)

Re: Looping Activiti Workflow

filipearaujo — Thu, 28 Jul 2011 18:42:20 GMT

Yes I understand the solution but I need to run multiple loops not only one, if for all loops I've to make something like that it is really bad.

Thanks for the replies.

Re: Looping Activiti Workflow

coner — Thu, 28 Jul 2011 19:14:17 GMT

agreed…. but we essentially have no other choice, as far as the developers have told me / the forums

do you want parallel loops in a single workflow? or in multiple different workflows executing in parallel, each with a loop?

it is easier if you change parallel loops in a single workflow into the case where each loop is its own workflow. you can then use a multi-threaded activiti-controller to invoke each of the loop items. This would have to be a blocking loop call that would allow return values from the loop to obtained most likely.

Re: Looping Activiti Workflow

ronald_van_kuij — Thu, 28 Jul 2011 21:26:13 GMT

there are several statements in replies in this topic that I'd like to address. Some are wrong and some are the result of assumptions that how things could be done by diving directly into technical solutions instead of looking at it from a functional standpoint and asking for solutions for this.

execution stack dumps are not acceptable when the workflow may not actually contain recursion or infinite loops.

That is tuning… If I give my JBoss just 128M, it crashes all the time with oom without loops. We need to do things like tuning all the time…

i want to make a loop with a condition that it may take long or short time, and it gives me the "StackOverflow" exception.

Long running is not the problem, the number of loop BEFORE a waitstate is encountered is..

short version:
no.

Yes and no. Yes if there is a wait-state, no if there are to many repetitions and tuning is not done correctly (tuning can't solve all)

long version:
this is because memory is being added to the stack at every step inside of your workflow (by the engine), never being cleaned up, it will eventually cause a stack overflow.

I did not say that memory was never released, it is not released until a waitstate is encountered. This is because a commit to the db still needs to take place… Keeping all things in memory requires well… memory. And not per workflow, per engine

they would prefer that you use an external system that will periodically execute the workflow. this would avoid the memory issue.

That is your interpretation based on your first remark in the first post in this forum

I have been trying to implement a daemon process that will wakeup and do some work every X time in activiti.

What I said use an external system to execute the service and have a waitstate in the process. This could e.g. be done by a minimal service task that puts a message in a jms queue somewhere, followed by a receive task (wait state! so persistence takes place, so memory is freed) with a timeout on it that makes it continue/loop… After the job finishes normally, the receive task is triggerd. This is way simpler than your example and can even be achieved plain jms or with combining e.g. Mule and Activiti or [url=http://fox.camunda.com/peal-psi/Camunda PSI

Oh and Async continuations would be another option

agreed…. but we essentially have no other choice, as far as the developers have told me / the forums

Totally disagree here… No one told you that 'your solution' was the only choice as you can read above… Lots of other choices, so please do not draw wrong conclusions and pretend we said things…

Re: Looping Activiti Workflow

coner — Thu, 28 Jul 2011 22:36:34 GMT

isn't forcing people who only want to use service tasks into using wait states an unnecessary requirement?

i'm gonna ignore the rest and jump to the java receive tasks, which I had not read about until now.

probably the only semi viable solution is in the Java receive task's. but is the only way to wake someone that is in the wait state is by:


ProcessInstance pi = runtimeService.startProcessInstanceByKey("receiveTask");
Execution execution = runtimeService.createExecutionQuery()
  .processInstanceId(pi.getId())
  .activityId("wait")
  .singleResult();
assertNotNull(execution);
    
runtimeService.signal(execution.getId());

correct? so i have to identify the process by name.

this is ok for simple tasks that essentially bounce back and fourth between each other…

i am dynamically deploying these workflows in a multi-tenant system which is using JMS for unique and private communication between workflow java tasks and remote java class execution. (this reduces the memory / compute footprint required by the tasks within the activiti engine since they only pass JMS now)

hence my complicated example that i provided before.

runtimeService.signal(execution.getId());
this does not allow any dynamic parameter passing to the process that we are telling to resume correct? or is there some other way to pass parameters via this method?

Re: Looping Activiti Workflow

ronald_van_kuij — Thu, 28 Jul 2011 23:39:10 GMT

isn't forcing people who only want to use service tasks into using wait states an unnecessary requirement?

No, it's called working within the posibilities of a framework/… I for intance want to be able to write a multi-milion application in java in one line of code… Can't be done…

i'm gonna ignore the rest and…

Confronting isn't it?

i'm gonna ignore the rest and jump to the java receive tasks, which I had not read about until now.

Receive tasks, not JAVA receive tasks… and I mentioned them before[

quote="COner"]correct? so i have to identify the process by name.
No, the process INSTANCE, and by ID

this is ok for simple tasks that essentially bounce back and fourth between each other…

i am dynamically deploying these workflows in a multi-tenant system which is using JMS for unique and private communication between workflow java tasks and remote java class execution. (this reduces the memory / compute footprint required by the tasks within the activiti engine since they only pass JMS now)

Sorry, I do not understand why this is only for simple tasks etc… Can you elaborarte? And yes, maybe your 'solution' is a complicating factor because of the way it is implemented… You always need some kind of 'correlation' and the combination of the process instance id and the name of the receive task to be called is an perfectly normal example in lots of businesscase

hence my complicated example that i provided before.

Again, sorry… I miss the reasoning behind the 'hence'…

this does not allow any dynamic parameter passing to the process that we are telling to resume correct?

No clue what you mean by 'dynamic parameter passing'… You seem to talk in technical terms a lot, what is it that you want to achieve? Do you want values that come from the external service to be added to the specific process instance?