Process variable content not properly passed to subprocess

[waldo]

Dear community,

I am stuck in a problem related to parameter passing and asynchronous subprocess calls and I hope that somebody can help me

We are evaluating the use of Activiti (5.11) in a current project. We created some processes (which I cannot attach here) to get acquainted with Activiti.

The processes use Activiti process variables to pass information around. The data in these variables is supposed to be "threaded through", i.e.
on each Call Activity, these parameters are passed in (via input parameters, e.g. source=X, target=X) and returned to the caller
(via output parameters, e.g. source=X, target=X).

We experience the following strange behaviour: the value of some variable X (a large Serializable object) for a subprocess executed through a
CallActivity does NOT contain the value of X as it is when the calling process calls the subprocess, but instead it contains the value of X as it was when the calling process itself started its execution.

There are 4 process definitions involved:

Syn (S)
Incoming (I)
Operate (O)
Confirm (C)

The execution sequence is as follows (service tasks executed in between are omitted, only calls are shown):

- O is started by some external "event", i.e. in Java code
- The value of X.foo.bar.baz is 0 [X.foo.bar.baz==0]
- O calls S
- S starts
- S waits for a message (message intermediate catching event)
- I is started by some external "event", i.e. in Java code
- I sends the message that S is waiting for
- S receives the message and continues
- meanwhile, I ends
- S reaches its end event, we still have [X.foo.bar.baz==0]
- O is resumed, still [X.foo.bar.baz==0] holds
- O's service tasks modify X, setting X.foo.bar.baz to 1
- O calls C
- C's first service task (right after the start event) detects [X.foo.bar.baz==0] and fails (it expects it to equal 1)

I tried to debug this problem myself, but honestly, this was hopeless for me given the way that Activiti handles process execution. The stack was ~1700 items
deep(!), mostly containing calls to CommandContext.performOperation, ExecutionEntity.performOperation and others.
I have implemented more than one finite state machine myself, and it never turned out to be a good idea to use the OS/VM stack for maintaining the machine's state.
I'm really curious why you chose this design. Any background information would be highly appreciated.

Btw: At one point, we even encountered a StackOverflowError, which we fixed by adding some more async…

We use Oracle 11g as a DataSource, the job executor is activated, all CallActivities are set to asynchronous (activiti:async="true")

Many thanks for any hints!

Regards,
Dennis

[jbarrez]

It's pretty hard to say with only this description of course, but

The stack was ~1700 items deep(!)

That for sure doesn't sound right.  We do have a mechanism in place to avoid using the stack, but from what I read it might be broken here. How many steps does the process contain once you trigger it. Also keep in mind that the transaction will be open for the whole duration of all steps… some database (Oracle definitely) don't really like that.

[waldo]

Unfortunately I'm not allowed to upload process definitions.

Some more info:

Basically all our activities are Java Service Tasks configured via delegate expressions, often times with start listeners attached:

    <serviceTask id="prepareAuthentication" name="prepareAuthentication" activiti:delegateExpression="${nd_PrepareAuthentication}"></serviceTask>
   
    <serviceTask id="setStateProcessing" name="setStateProcessing" activiti:delegateExpression="${nd_SetState}">
      <extensionElements>
        <activiti:executionListener event="start" class="com…..ServiceTaskListener">
          <activiti:field name="parameterName">
            <activiti:string>state</activiti:string>
          </activiti:field>
          <activiti:field name="parameter">
            <activiti:string>IN_PROGRESS</activiti:string>
          </activiti:field>
        </activiti:executionListener>
      </extensionElements>
    </serviceTask>

Numbers of service tasks: Incoming 13, Operate 61, Confirm 21, Syn 5
Processes generally contain loops, but in the case described no loop is executed more than once. All service tasks have exactly 1 outgoing connector into an Exclusive Gateway that dispatches on the result of the node. So roughly, there are as many Exclusive Gateways as there are service tasks (merge gateways not counted).

Call activities are defined thus:

    <callActivity id="ConfirmationRequestCreateAndPublish" name="ConfirmationRequestCreateAndPublish" activiti:async="true" calledElement="ConfirmationRequestCreateAndPublish">
      <extensionElements>
        <activiti:in source="PAYLOAD" target="PAYLOAD"></activiti:in>
        <activitiut source="PAYLOAD" target="PAYLOAD"></activitiut>
      </extensionElements>
    </callActivity>

Processes are started using

org.activiti.engine.RuntimeService.startProcessInstanceByKey(String, String, Map<String, Object>)

The message that is being sent from Incoming to Operate is realised via:

org.activiti.engine.RuntimeService.messageEventReceived(String, String, Map<String, Object>)

I'd like to attach the stack trace and Spring configuration, but the forum doesn't let me.

Any help would be highly appreciated, as the huge stack is really a show-stopper for us.

[jbarrez]

Numbers of service tasks: Incoming 13, Operate 61, Confirm 21, Syn 5
Processes generally contain loops, but in the case described no loop is executed more than once. All service tasks have exactly 1 outgoing connector into an Exclusive Gateway that dispatches on the result of the node. So roughly, there are as many Exclusive Gateways as there are service tasks (merge gateways not counted).

Those numbers aren't really high … that should be no problem. Like I said, it was designed to cope with that, but I'm feeling something might went wrong there …

Any help would be highly appreciated, as the huge stack is really a show-stopper for us.

It's pretty hard to test without a test case. But I'll to to reproduce it myself locally.
Also, please do take in mind what I said about long transactions. That's why we added async continuations. I've seen plenty of DBA's going wild over keeping open transactions for a long time.

[waldo]

I analysed Activiti stack behavior further and I am surprised by its stack consumption.

The simplest looping process consisting of a single synchronous service task increasing process variable N and one gateway checking N:

Start -> ServiceTask -> Gateway
Gateway [N >= 1000] -> End
Gateway [N < 1000] -> ServiceTask

representing a loop like this:

while (N < 1000) do
  N = N + 1 // ServiceTask

exits with a StackOverflowError after only 114 iterations. In every iteration the stack grows by roughly 80 elements!

Can you provide some details about the mechanism to avoid using the stack? How can I enable it, except by using async tasks, which would increase DB interaction quite drastically?

[jbarrez]

Hmmm that sounds scary and buggy. When that piece was implemented (int he atomic operations) it wasn't the case, I checked it back then.

I'll look into it and see if there are workarounds/fixes that quickly can be made