Is there way to limit number of concurrent process instances?

[joeysr20det]

Is there a way to limit the number of concurrent process instances the engine can run? If so, is there any type of queuing mechanism that backlog process instances to run them later (when the concurrent limit is no longer exceeded)?

I've done a good deal of searching both the net and guide as well as digging through source code and I haven't found anything. I apologize if in fact this has already been answered and I have just missed it.

We were just wanting to check and make sure this type of a functionality didn't already exists before we went and built it on top of our implementation; didn't want to reinvent the wheel It just seems as though there would be some mechanism for this dealing with this already.

Thanks to anyone that has any input and thank you to the Activiti Devlopers for such an awesome engine. This thing rocks!

[martin_grofcik]

Hi,

I do not see the reason why you need to limit concurrent process instances. Could you provide use case?

There is a possibility to control concurrent process instances in async execution.
http://activiti.org/userguide/index.html#jobExecutorConfiguration

Regards
Martin

[joeysr20det]

Sure. So we implemented the use of Activiti within our Document Management system several years ago. Additionally, we've created what we call Workflow Process Triggers. The idea behind these triggers is that I can say every time a document of a specific nature is created, modified, etc. run this specific process. While this triggers work very well at triggering a workflow sequence for all sorts of events, we do have a few corner cases we are looking to deal pertaining to bulk operations.

For instance, if I have the need to execute a process every time a specific type of document is created. While this no issue at all for small operations (of one, 10, 20, 30 documents), we run into issues when users bulk load thousands, tens of thousand, etc. While this is zero issue on the rest of our system, it's pretty easy to bring down the workflow engine.

Our idea is that if available we could configure the engine to say accept up to a thousand, ten thousand, etc. concurrent processes (whatever a reasonable limit is for the given server) and then begin to backlog the start process instance requests until the engine falls back under that threshold.

We've done a good deal of searching and don't see anything for this type of need in the Activiti user guide, source code, etc. While we can easily add this to our software, we just wanted to make sure nothing like this existed already as to not reinvent the wheel

Thanks for the quick reply.

[martin_grofcik]

Hi,

From my point of view, it is possible to execute bulk operations asynchronously. In a case of async operations you can implement  job executor and control throughput in it.

Regards
Martin

[joeysr20det]

Thanks for the input. We'll definitely look into that.

Are there any sort of specifications we could refer to anywhere as to safe/reasonable number of concurrent process instances for say a given amount of JVM memory? I know it's all relative as to the size and impact of the process definitions themselves, but we wanted to make sure we hadn't looked over or missed any kind of "rule of thumb" advice in our searching.

[jbarrez]

No, there is no real rule of thumb here. Typically, you'll be limited by the database and the database connections.
The way  read your use case, I would use the new Async executor. A few thousand async process instances should not bring down the engine.