Possible bug in eventing - Activiti not ready to receive signal even after dispatching ACTIVITY_STARTED for signal catch node

[michalwrobel]

I've been experimenting recently with events in Activiti 5.15.
I developed a process which has some tasks and intermediate signal catching events
and a client app which sends these signals after receiving ACTIVITY_STARTED event for appropriate signal catching node.

The problem is Activti doesn't always seem ready to receive the signal in the moment when ACTIVITI_STARTED event has been already dispatched.

Important notes :
- I haven't been able to reproduce this bug using activiti embedded in unit test on a simple process
- Activiti engine works inside webapp (the standard activiti REST war), which I extended to publish events using websockets. The client sends signals using REST API (POST runtime/signals)

Here's the diagram of the part of process relevant to this post :
[img]https://dl.dropboxusercontent.com/u/1846654/act.png[/img]


And some simplified pseudocode of what happens in client app :

processInstanceId = createAndStartProcessInstance();  // performs all steps leading to op1-finished signal catching node
waitForEvent("op1-finished", "ACTIVITY_STARTED");   // it will only continue ONLY IF ACTIVITY_STARTED for op1-finished catching node is received, so node should be ready to receive signal   
//  sleep(300);   //when i uncomment this it ALWAYS work
signalOperation1Finished(); // this always succeeds but whithout sleep the signal isnt always received properly
…
checkIfProcessFinished(processInstanceId);  // fails often without 'sleep' because signal is sent 'too early'(?) and process hangs at waiting for signal 
‍‍‍‍‍‍‍‍

Interesting thing is that without 'sleep' signal fails to be delivered almost always just after starting webapp server and then after some repetitions of client test tends to succeed often, just like activiti engine underneath needs some kind of 'warmup' before it can perform in time.

The ACTIVITY_STARTED event is the last event sent by engine before process stops on event catching node so I have to send signal after receiving it, isn't it?
Quote from the documentation caught my eye "ACTIVITY_STARTED   An activity is starting to execute"
IS STARTING, not 'have been started' or something like that so… is it a bug? Or is some later event still lacking in the API?
When I can be sure the process node is ready to catch signal?
Obviously I cannot use 'sleeps' in production code..

So..
- Have I found lacking feature or a bug? Is really engine not exactly ready to process signal in the exact moment of dispatching 'ACTIVITY_STARTED' event?

[jbarrez]

The problem is most likely that the changes have not been comitted to the database yet. Only when it's committed, you can be sure it's ready.

Im not sure what the way to handle thay would be… would a new event be needed so you can listen to 'commit' and 'flush' events?

[michalwrobel]

So.. you confirm problem exists.
Do you have any hint how it can be resolved for now? (Waiting for commit/flush 'events' ? )
Do you plan developing new kind of events for 'commited activity_started' ?

[frederikherema1]

The events are low-level events and should be treated with care, in case you want to manipulate the process the event originated from, using the API (eg. signal).

At the time the ACTIVITY_START is fired, the transaction that is done in is not yet committed, nor are the changes flushed to the DB. This means that it could be that the transaction is rolled back later on and the event is invalidated. This is something users should handle themselves e.g:

- Using a transaction-synchronisation adapter to actually execute the action needed as a result of a specific event AFTER the state of the process has been persisted to the DB.
- Participating in the transaction activiti uses for any DB related servicecalls you may do, so that it's also rolled back.

The main question is WHY you are using these low-level events in a process to direct process-flow? This is not the main usecase for these kind of events, better constructs in BPMN are available for that, imho (depending on your usecase).

[michalwrobel]

Well uhm.. now I'm confused. You're basically saying my understanding of purpose of eventing system is wrong. Can you then give an example of 'desired usecase' for ACTIVITY_STARTED event (and the others )? 
What 'real life action' could I take based on information this ACTIVITY_STARTED event carries? ('hi it's your process engine , well i'm setting up the node, it should be ready in SOME time , but uhm you know it also can be destroyed forever if I happen not to commit my transaction..' )

Why am I using this? Let's go back to the image in my first post in this thread.
What is the other way to be 100% sure that process is now waiting for op1-finished signal not using events?
Did I miss something and it's possible via (REST)API?
But even if that is possible via some REQUESTS it would be polling, not eventing. Polling consumes resources and also lead to some delays and these both factors are really not desirable in my use case.
I would like to get notified just-in-time that the 'node is ready' - how is it possible without eventing?
Would it be hard to develop kind of events I need, for 'commited states of process' ?

[frederikherema1]

Some of the 'real life' usecases I can think of are: logging purposes, validation purposes (throw exception when some condition is not met), metrics gathering, …

The event system is what it is: throwing raw events when they occur. If you want to have a nice way of doing stuff with the process-scope (e. setting variables, getting variables) you should take a look at the executionListeners on process-definition level. But even then, the executionlisteners are called when the transaction is not yet committed. This is just the way activiti works with transactions, there is no way around it (as with any other framework that exposes events as part of a non-comitted transaction).

As I suggested in the post above, you CAN leverage the events to do this IF you add a transaction-listener that does whatever you need to do. So instead of instantly calling the action you want to do (signal), add a transaction listener that does this instead.

Use a transaction synchornisation adapter if you're using spring. You can also hook into the listener-mechanism that activiti uses (eg. to decrement job retries) to be sure it runs on al transaction-environments activiti supports (implement org.activiti.engine.impl.cfg.TransactionListener):


TransactionContext transactionContext = Context.getCommandContext().getTransactionContext();
    transactionContext.addTransactionListener(TransactionState.COMMITTED, yourImplementation);

Please note this is in the impl package.

[michalwrobel]

Ok, I will then have to check these transactions listeners, thanks for your help and putting me on the right path

[michalwrobel]

Thanks for the hint about transaction listeners - it works like a charm now