topic Re: Queue and process monitoring in Alfresco Archive

Queue and process monitoring

gromar — Tue, 09 Oct 2012 16:05:11 GMT

Hello all.I am thinking about process instances monitoring features, application of queuing theory in process optimization. The basic thing to monitor can be queue of waiting process instances in waiting states (e.g. user task). Currently there is possibility to add timer on user task. It could be i

Re: Queue and process monitoring

frederikherema1 — Wed, 10 Oct 2012 07:39:49 GMT

That seems like a quite specific use-case, inspecting history of completed instances (this can get specific and complex, based on needs). IMHO, this is something you could easily add in top of the Activiti API (all building blocks are available).

Re: Queue and process monitoring

nbuso — Wed, 17 Oct 2012 08:59:18 GMT

Sorry I can't understand, I think is quite a need to check what happen to a run processes. Is there any "visual" way to monitor a finished processes? I really need monitoring of processes from the GUI activiti is not offering such functionality?

Re: Queue and process monitoring

frederikherema1 — Wed, 17 Oct 2012 11:05:24 GMT

Activiti Explorer does not offer any visual "monitoring" of a process-instance other than a process-diagram of the process and the current active activities highlighted. For finished processes, all information is available in the history and can be retrieved through our API but there is NO GUI for that in activiti explorer.

Re: Queue and process monitoring

gromar — Wed, 17 Oct 2012 15:14:40 GMT

That seems like a quite specific use-case, inspecting history of completed instances (this can get specific and complex, based on needs). IMHO, this is something you could easily add in top of the Activiti API (all building blocks are available).

I think you are right, everything is there and on the top of API (may be with small changes) monitoring can be implemented.
I can be more specific now. I established a project on github to provide possibility to monitor running instances of the process.
http://gro-mar.github.com/activiti-crystalball

https://github.com/gro-mar/activiti-crystalball.git

and wiki
https://github.com/gro-mar/activiti-crystalball/wiki

(I would appreciate any comment suggestion…..)

Let's imagine:
1. there is a queue of tasks assigned to one actor. (This actor is dedicated to complete only these tasks in the workflow)
2. there are already 5 tasks in the user task queue, There is timer signal assigned to user tasks (withdraw task after 10mins in the queue).

What I want to achieve:
Monitor queue and throw a signal in the case when probability that timer signal on the user task will be thrown is higher than 80%.

It is only school example. I do not know whether it has application in real life.
To calculate probability simulation can be used (I think activiti does not support simulation yet).
Throwing a signal in the case of reaching 80% limit is simple.

What is your opinion:
Does it make sense to base monitoring on monitoring queues?

Re: Queue and process monitoring

jbarrez — Thu, 18 Oct 2012 09:38:04 GMT

In my opinion, a queue will make development harder.

I did a prototype once, where I had very simple task and execution listeners attached to my process.
In those listeners, I fired events to some queue that would process these events and generate monitoring information.
Adding such listeners is very easy in Activiti, and you don't need to add them to the process xml: just configure a BPMNParseListener that attaches this listener to any of the workflow steps.

Re: Queue and process monitoring

gromar — Thu, 18 Oct 2012 12:38:17 GMT

Thank you for answer

just configure a BPMNParseListener that attaches this listener to any of the workflow steps

Thats good point - I will do that.
Process model does not contain information about queue sizes (they can differ from node to node). This information can be stored somewhere.

But it is interesting:
Another idea is to use simulation for event prediction in the system.
The first idea -
1. take current state of the system (all running process instances)
2. parse deployed process definitions and change node implementation. E.g. for user task we can simulate user work, and generate complete events to the simulation event calendar, … .
3. execute simulation runs several times, and generate report.

Can BPMParseListener implementation be used to completely change node behavior?

Re: Queue and process monitoring

jbarrez — Fri, 19 Oct 2012 05:46:11 GMT

The BPMNParseListener can tweak node behavior to a certain extent, yes. But you will need to learn some of the internals of course.

What exactly would you like to change in step 2? What would be the output of a simulation run then?

Re: Queue and process monitoring

gromar — Fri, 19 Oct 2012 08:20:49 GMT

What I want to achieve:
just one simple case
Monitor system (running process instances). In case when probability of expiring user task in the the queue will be higher than limit (e.g. 80%), generate report, inform somebody about this state
For simplification take process with
. [start]—>[user task]—>[end]
. [signal time expired]
User task is assigned to the one user.

How
1. Take current state of the system (running process instances, taskService tasks, userService configuration (users)). To create simulation model.

What exactly would you like to change in step 2?

2. change behavior of the user task and other nodes.
Simulator will simulate actor's work. It generates events to the simulator event calendar. - CompleteTask on which actor is currently working on, event to claim next task …..
Event calender will contain periodical events to execute jobs (check task duration and signal). When signal should be thrown - it will throw a signal and log it into simulator audit trail.
Simulation will stop when there will be no user task in the system.

In the first step I would like to use process engine to react as a model and will be driven by simulation time. This process engine should work on the top of another DB scheme - to do not touch real world state, but to work on simulated data.

What would be the output of a simulation run then?

In that case output of simulation run will be audit log. this audit log has to be evaluated whether something interesting has happened (e.g. duration signal).
Output of one simulation is process audit trail.
Simulation is repeated 10t times, and in the case when user task expires in 8t cases we can say that there is 80% that duration will exceed.
Simulation output has to be evaluated - we can calculate probability whether user task will expire.

Report from simulation is process diagram with highlighted signal node on the user task and with probability description(80%)

Does it make sense?
Is it possible to do it?
How can it be done with activiti?

Re: Queue and process monitoring

jbarrez — Mon, 22 Oct 2012 09:34:32 GMT

Sure it makes sense, but I'm just not sure if the simulation engine also should be the activiti engine …

Everything you state can be done by introducing listeners (execution / task / process) to the process.
Like I said, this can be done transparantly by using a BPMN parse listener.
Just fire off the events to whatever queue you have and process them.

Re: Queue and process monitoring

gromar — Thu, 15 Nov 2012 16:16:22 GMT

The simplest DueDate monitoring results should look like:

[img]https://github.com/gro-mar/process-monitor/blob/timer-prediction/generator/src/test/resources/org/processmonitor/generator/usertasksimulator/TaskInstanceDueDateMoniorTest.dueTime1_50Reached.png?raw=true[/img]

It means that in 100% of simulation runs due time for at least one task in the queue for checkFormalRequirements node was met and in 61% of simulation runs due time for ValueCheck was met.

(there is a test which has generated this process diagram https://github.com/gro-mar/process-monitor/blob/timer-prediction/generator/src/test/java/org/processmonitor/generator/usertasksimulator/MockTaskInstanceHistoryExecutorTest.java)

Question
Does HistoricTaskInstance


  /** Difference between {@link #getEndTime()} and {@link #getStartTime()} in milliseconds.  */
  Long getDurationInMillis();

return time delta between claiming the task and task complete/delete?

Another topic
Next time I would simulate more ( users in the group,…) Timers, gateways decisions…..
How can I get potential task owners from the Task?

Re: Queue and process monitoring

jbarrez — Thu, 15 Nov 2012 19:26:53 GMT

return time delta between claiming the task and task complete/delete?

No, the time between the task creation time and when it is completed

How can I get potential task owners from the Task?

Check the candidateUsers / candidateGroups attribute on a user task.

Re: Queue and process monitoring

gromar — Fri, 16 Nov 2012 13:03:14 GMT

Duration
<code>
getDurationInMillis();
</code>
it make sense to get only task creation time and task end time in the process audit trail.

From logical point of view task claim/complete event should be part of TaskService audit trail. Does something like this exist, from where I can get this kind of information?
How can I extend TaskService to log this information?

Candidate users
org.activiti.engine.task.Task Interface (Returned from the TaskInstanceQuery) does not have any getCandidateUsers/groups method. In fact taskQuery is returning List<Task>, which is TaskEntity which implements TaskDelegate interface. (To get candidates I can recast it - not clean) - Is there better way?

Re: Queue and process monitoring

jbarrez — Fri, 16 Nov 2012 18:23:24 GMT

From logical point of view task claim/complete event should be part of TaskService audit trail. Does something like this exist, from where I can get this kind of information? How can I extend TaskService to log this information?

No, this information is indeed not recorded.
To fix this, you'd need to extend the current ClaimTaskCmd and add a call to the history manager yourself.

org.activiti.engine.task.Task Interface (Returned from the TaskInstanceQuery) does not have any getCandidateUsers/groups method. In fact taskQuery is returning List<Task>, which is TaskEntity which implements TaskDelegate interface. (To get candidates I can recast it - not clean) - Is there better way?

Yes - use the getIdentityLinksForTask method on the TaskService

Re: Queue and process monitoring

gromar — Wed, 21 Nov 2012 14:48:17 GMT

No, this information is indeed not recorded.
To fix this, you'd need to extend the current ClaimTaskCmd and add a call to the history manager yourself.

I have implemented it locally. How can I publish these changes? (another branch, I do not have permissions, is there any need to have such a functionality ….)

Re: Queue and process monitoring

jbarrez — Thu, 22 Nov 2012 12:38:22 GMT

Since we're on GitHub, the easiest way is just forking our project and creating a pull request (like all the cool frameworks nowadays do ;-))