Queue and process monitoring

[gromar]

Hello all.

I am thinking about process instances monitoring features, application of queuing theory in process optimization. The basic thing to monitor can be queue of waiting process instances in waiting states (e.g. user task). Currently there is possibility to add timer on user task. It could be interesting to have possibility to withdraw waiting in the queue or create a signal when there is high probability that task won't be finished on time (can be based on task execution history).
Queues can detect e.g. bottlenecks of the process (when they are monitored).

Why process definition does not support queues?

[frederikherema1]

That seems like a quite specific use-case, inspecting history of completed instances (this can get specific and complex, based on needs). IMHO, this is something you could easily add in top of the Activiti API (all building blocks are available).

[nbuso]

Sorry I can't understand, I think is quite a need to check what happen to a run processes. Is there any "visual" way to monitor a finished processes? I really need monitoring of processes from the GUI activiti is not offering such functionality?

[frederikherema1]

Activiti Explorer does not offer any visual "monitoring" of a process-instance other than a process-diagram of the process and the current active activities highlighted. For finished processes, all information is available in the history and can be retrieved through our API but there is NO GUI for that in activiti explorer.

[gromar]

That seems like a quite specific use-case, inspecting history of completed instances (this can get specific and complex, based on needs). IMHO, this is something you could easily add in top of the Activiti API (all building blocks are available).

I think you are right, everything is there and on the top of API (may be with small changes) monitoring can be implemented.
I can be more specific now. I established a project on github to provide possibility to monitor running instances of the process.
http://gro-mar.github.com/activiti-crystalball

https://github.com/gro-mar/activiti-crystalball.git

and wiki
https://github.com/gro-mar/activiti-crystalball/wiki

(I would appreciate any comment suggestion…..)

Let's imagine:
1. there is  a queue of tasks assigned to one actor. (This actor is dedicated to complete only these tasks in the workflow)
2. there are already 5 tasks in the user task queue, There is timer signal assigned to user tasks (withdraw task after 10mins in the queue).

What I want to achieve:
Monitor queue and throw a signal in the case when probability that timer signal on the user task will be thrown is higher than 80%.

It is only school example. I do not know whether it has application in real life.
To calculate probability simulation can be used (I think activiti does not support simulation yet).
Throwing a signal in the case of reaching 80% limit is simple.

What is your opinion:
Does it make sense to base monitoring on monitoring queues?

[jbarrez]

In my opinion, a queue will make development harder.

I did a prototype once, where I had very simple task and execution listeners attached to my process.
In those listeners, I fired events to some queue that would process these events and generate monitoring information.
Adding such listeners is very easy in Activiti, and you don't need to add them to the process xml: just configure a BPMNParseListener that attaches this listener to any of the workflow steps.

[gromar]

Thank you for answer

just configure a BPMNParseListener that attaches this listener to any of the workflow steps

Thats good point - I will do that.
Process model does not contain information about queue sizes (they can differ from node to node). This information can be stored somewhere.

But it is interesting:
Another  idea is to use simulation for event prediction in the system.
The first idea -
1. take current state of the system (all running process instances)
2. parse deployed process definitions and change node implementation. E.g. for user task we can simulate user work, and generate complete events to the simulation event calendar, … .
3. execute simulation runs several times, and generate report.


Can BPMParseListener implementation be used to completely change node behavior?

[jbarrez]

The BPMNParseListener can tweak node behavior to a certain extent, yes. But you will need to learn some of the internals of course.

What exactly would you like to change in step 2? What would be the output of a simulation run then?

[gromar]

What I want to achieve:
just one simple case
Monitor system (running process instances). In case when probability of expiring user task in the the queue will be higher than limit (e.g. 80%), generate report, inform somebody about this state
For simplification take process with
.  [start]—>[user task]—>[end]
.            [signal time expired]
User task is assigned to the one user.

How
1. Take current state of the system (running process instances, taskService tasks, userService configuration (users)). To create simulation model.

What exactly would you like to change in step 2?

2. change behavior of the user task and other nodes.
Simulator will simulate actor's work. It generates events to the simulator event calendar. - CompleteTask on which actor is currently working on, event to claim next task …..
Event calender will contain periodical events to execute jobs (check task duration and signal). When signal should be thrown - it will throw a signal and log it into simulator audit trail.
Simulation will stop when there will be no user task in the system.

In the first step I would like to use process engine to react as a model and will be driven by simulation time. This process engine should work on the top of another DB scheme - to do not touch real world state, but to work on simulated data.

What would be the output of a simulation run then?

In that case output of simulation run will be audit log. this audit log has to be evaluated whether something interesting has happened (e.g. duration signal).
Output of one simulation is process audit trail.
Simulation is repeated 10t times, and in the case when user task expires in 8t cases we can say that there is 80% that duration will exceed.
Simulation output has to be evaluated - we can calculate probability whether user task will expire.

Report from simulation is process diagram with highlighted signal node on the user task and with probability description(80%)

Does it make sense?
Is it possible to do it?
How can it be done with activiti?

[jbarrez]

Sure it makes sense, but I'm just not sure if the simulation engine also should be the activiti engine …

Everything you state can be done by introducing listeners (execution / task / process) to the process.
Like I said, this can be done transparantly by using a BPMN parse listener.
Just fire off the events to whatever queue you have and process them.