topic Re: HPC for BPM /Business Rules -- Support in Activiti in Alfresco Archive

HPC for BPM /Business Rules -- Support in Activiti

k_kunti — Tue, 09 Oct 2012 06:42:15 GMT

Dear AllI have been thinking implications of High performance computing in BPM specifically in the area of Business rules where data size and rule complexity might be significant. I feel the capability to run complex business rules on significant amount of data (in GB/TB) will make adoption of BPM i

Re: HPC for BPM /Business Rules -- Support in Activiti

jbarrez — Thu, 11 Oct 2012 14:08:17 GMT

What exactly are you looking for in a discussion?

I mean, of course we have thought already about map reduce and big data and how it related to bpm … but so far the use cases for BPM don't quite match those of big data. Business processes need to be strict and just in time, not 'eventual consistent'.

Maybe I'm not seeing the picture you're trying to sketch … to feel free to elaborate.

Re: HPC for BPM /Business Rules -- Support in Activiti

k_kunti — Mon, 15 Oct 2012 11:15:38 GMT

Hi barrez

Thank you for replying, my thoughts are following direction:

1. Achieving high performance by parallelizing processing of certain steps , where human participant mentions the task to be executed in parallel (by System - e.g Parallel_Business_Rules_task or sending Mails). The BPM engine takes care of processing the same task on multiple nodes of BPM cluster and joins back to the originator node on completion. Where HDFS kind of infrastructure is used to take care of file replication/consolidation across these nodes. These tasks generally work on file or DB (relation or column based).

2. Essentially the above approach the user decides which business step can run in parallel (say sending mail to 1,00,000 users) and what part of data each of these parallel braches work (say 10,000 mails per BPm node ) is decided by BPM engine based on some provided threshold. By the virtue of running BPM engine across multiple node we could possible shift execution of a task to multiple BPM engines in the cluster provided we have required data on those nodes.

3. This would mean having some kind of Job monitor and failure recovery mechanism. Wherein we can think of Activiti GRID engine (the new software piece) to take care of such tasks, we can possibly borrow from hadoop kind of platform for such capability (not sure though).

One might reason that these are batch oriented tasks and hence better kept outside tradition BPM paradigm , but personally I see lot of value if a BPM engine can model and take care of such tasks (at a very high level to start with , say what the task does can be done in Plain Java, the platform takes case of rest of the management stuff)

Thank you
Best Regards
Krishnendu

Re: HPC for BPM /Business Rules -- Support in Activiti

ronald_van_kuij — Mon, 15 Oct 2012 17:33:30 GMT

One might reason that these are batch oriented tasks and hence better kept outside tradition BPM paradigm

Yep…

but personally I see lot of value if a BPM engine can model and take care of such tasks

I don't… Since it over complicates the bpm engine I think to achieve things like this. But that the are just my €0.02

Re: HPC for BPM /Business Rules -- Support in Activiti

jbarrez — Tue, 16 Oct 2012 09:57:11 GMT

I agree with Ronald here: it shouldn't be part of the engine.
There are specialized open source frameworks to achieve this: I'm now thinking of Spring batch, Spring integration, Apache Camel, JBoss Infinispan, etc…

One thing I plan to look into in the near future is making integration with these frameworks better in Activiti. As Krishnendu says, it should be easier to fire of tasks that require heavy computing. What we need to improve there first of course, is correlation in the engine. Now we only have the execution id. But it should be possible to correlate based on process data instead of these execution ids.