Job Acquisition filter

[smirzai]

We have a need here. In a cluster deployment model, it might be the case, that some nodes do not have the resource to do an async job, temporarily or permanently. The desired behavior would be to able to configure which nodes pick which jobs.

It might be also in some other case. Just imagine we have two machines, one fast and one slow and we want the higher priority tasks to be executed in Fast machines.

It is already mentioned in forum that it is not easy to implement the job prioritization by its sheer sense. But this way  might be a compromise.

Here is a proposal on configuration:
<bean id="jobExecutor" class="org.activiti.engine.impl.jobexecutor.JobExecutor">
   <property name="jobFilter" value="jobName='proc_def_1'"/>
</bean>

To address the above second case, it might useful to set a new generic parameter, say "param1" to job Executer table. This parameter can be set in xml definition file. Something like:
<serviceTask id="service1" name="Generate Invoice" activiti:class="my.custom.Delegate" activiti:async="true" activitiaram="1" <!– means to be run on fast machine/>

I can try to see how this feature could be implemented.  Would it be a reasonable extension ?

[frederikherema1]

We need to make sure this feature is as generic as possible and not break existing installs, which don't need the filtering at all. Adding an additional parameter (to both job-table and process-definition-xml) seems a bit too drastic. I like the idea more of an approach where the process-definition is used as a filter-parameter (or perhaps the job-type, eg. timers). This allows us to use the existing query-infrastructure to fetch jobs to execute and include those parameters.

Not sure what other community-members think about this but imho it's a valid addition to activiti…

[jbarrez]

You also could use the current job mechanism by creating your own JobHandlers, inject them into the process engine config.
These handlers have a database stored 'jobHandlerConfiguration' which you can use for whatever use you see fit.

[frederikherema1]

Joram, the problem with the jobHandler approach is that this is already too late. The job handler is called after the job has been acquired -> possibly on the wrong machine/cluster-node. To get a job back on the queue and hope for the correct machine to acquire it, is a lucky guess

[jbarrez]

Good point, Fred. Didn't think it through enough.

I'm just not a fan of meaningless parameters. In this case, I'd rather inject a custom JobExecutor than configuring it using a meaningless parameter.

[frederikherema1]

Adding meaningless parameters is indeed something we don't want, using the existing ones (handlerConfig, procDef, ..) never hurts. It can be implemented as a JobExecutor subclass that CAN (but not neccesairy must) be included in activiti. Since the executor is pluggable, this can be perfectly done outside of the activiti-code code.

[trademak]

Hi Saeid,

We want to implement functionality to do job prioritization for the 5.13 release.
In the coming days we'll update the Github wiki with more details about the ideas for this.
So it may be good to hold back any coding on this for the coming weeks and let's first discuss the implementation of this.
The wiki should be a good start for that.

Best regards,

[smirzai]

So my proposal would be changed to filtering but without general purpose parameter. It could be a custom implementation just with currently available fields.
Tijs, do you thing Job acquisition filtering will also be considered in job prioritization you just mentioned ?

[trademak]

The problem we see with the job filtering is that you define one job executor to take on a specific job.
When that job executor dies, the job will not be processed.
But you can always define a job filter in a custom implementation of a job executor.

Best regards,

[smirzai]

Hi tijs
The problem you mentioned is possible to occur, but it should not always be the case. You can define one main job picker which picks all the jobs  and another one which only services high priority jobs. The configurator should care, what he does. With great power comes great responsibility, Do you remember Spiderman ?
Or we can add another optional configuration parameter to job executor configuration, to allow job picker to pick a normal job, when it cannot find any job which matches the filter.
The good thing is that it could be configured so that, if you do not use it, it would be the same behavior as now.