topic Re: Job Acquisition filter in Alfresco Archive

Job Acquisition filter

smirzai — Mon, 18 Feb 2013 08:49:02 GMT

We have a need here. In a cluster deployment model, it might be the case, that some nodes do not have the resource to do an async job, temporarily or permanently. The desired behavior would be to able to configure which nodes pick which jobs.It might be also in some other case. Just imagine we have

Re: Job Acquisition filter

frederikherema1 — Mon, 18 Feb 2013 08:59:14 GMT

We need to make sure this feature is as generic as possible and not break existing installs, which don't need the filtering at all. Adding an additional parameter (to both job-table and process-definition-xml) seems a bit too drastic. I like the idea more of an approach where the process-definition is used as a filter-parameter (or perhaps the job-type, eg. timers). This allows us to use the existing query-infrastructure to fetch jobs to execute and include those parameters.

Not sure what other community-members think about this but imho it's a valid addition to activiti…

Re: Job Acquisition filter

jbarrez — Mon, 18 Feb 2013 09:07:59 GMT

You also could use the current job mechanism by creating your own JobHandlers, inject them into the process engine config.
These handlers have a database stored 'jobHandlerConfiguration' which you can use for whatever use you see fit.

Re: Job Acquisition filter

frederikherema1 — Mon, 18 Feb 2013 09:12:10 GMT

Joram, the problem with the jobHandler approach is that this is already too late. The job handler is called after the job has been acquired -> possibly on the wrong machine/cluster-node. To get a job back on the queue and hope for the correct machine to acquire it, is a lucky guess

Re: Job Acquisition filter

jbarrez — Mon, 18 Feb 2013 09:19:50 GMT

Good point, Fred. Didn't think it through enough.

I'm just not a fan of meaningless parameters. In this case, I'd rather inject a custom JobExecutor than configuring it using a meaningless parameter.

Re: Job Acquisition filter

frederikherema1 — Mon, 18 Feb 2013 09:35:01 GMT

Adding meaningless parameters is indeed something we don't want, using the existing ones (handlerConfig, procDef, ..) never hurts. It can be implemented as a JobExecutor subclass that CAN (but not neccesairy must) be included in activiti. Since the executor is pluggable, this can be perfectly done outside of the activiti-code code.

Re: Job Acquisition filter

trademak — Mon, 18 Feb 2013 11:04:18 GMT

Hi Saeid,

We want to implement functionality to do job prioritization for the 5.13 release.
In the coming days we'll update the Github wiki with more details about the ideas for this.
So it may be good to hold back any coding on this for the coming weeks and let's first discuss the implementation of this.
The wiki should be a good start for that.

Best regards,

Re: Job Acquisition filter

smirzai — Mon, 18 Feb 2013 16:01:58 GMT

So my proposal would be changed to filtering but without general purpose parameter. It could be a custom implementation just with currently available fields.
Tijs, do you thing Job acquisition filtering will also be considered in job prioritization you just mentioned ?

Re: Job Acquisition filter

trademak — Tue, 19 Feb 2013 09:40:11 GMT

The problem we see with the job filtering is that you define one job executor to take on a specific job.
When that job executor dies, the job will not be processed.
But you can always define a job filter in a custom implementation of a job executor.

Best regards,

Re: Job Acquisition filter

smirzai — Tue, 19 Feb 2013 14:24:57 GMT

Hi tijs
The problem you mentioned is possible to occur, but it should not always be the case. You can define one main job picker which picks all the jobs and another one which only services high priority jobs. The configurator should care, what he does. With great power comes great responsibility, Do you remember Spiderman ?
Or we can add another optional configuration parameter to job executor configuration, to allow job picker to pick a normal job, when it cannot find any job which matches the filter.
The good thing is that it could be configured so that, if you do not use it, it would be the same behavior as now.

Re: Job Acquisition filter

trademak — Wed, 20 Feb 2013 11:08:29 GMT

Hi Saeid,

I think the use case you are describing is perfectly fine, and it can be implemented with a custom job executor and the job priority configuration we are planning for 5.13.
But as far as I can see it now, there's no need to do additional things in the Activiti Engine.
So let's discuss this in relationship to the job priority configuration for 5.13. The first start of that is this wiki page

Best regards,

Re: Job Acquisition filter

jwohl — Fri, 22 Mar 2013 14:40:56 GMT

Hi there,
We've got a similiar problem in our development environment. There is a central database with the activiti schema. The application using the activiti engine runs on local websphere servers on each developers machine. For activiti and it's job aquisition it's like a distributed environment (it was even called cluster environment in a former post of this thread). Different developers work on the same process but implement different tasks. When they test the process in this kind of configuration the tasks are not necessarily executed on the developers machine who launched the process. Thats because how the standard Job Aquisition works. My question is: Could there be a special job aquisition filter which only selects the jobs which belong to the machine from where the process was instantiated? At the moment I can't see any proper selection criteria in the ACT_RU_* tables (i.e. in the ACT_RU_EXECUTION table). Maybe it would make sense to add a information like the node or machine name which instanciated the process to the datamodel and the Activiti Engine?
Thanks for your reply.
Best Regards,
Joern

Re: Job Acquisition filter

frederikherema1 — Fri, 22 Mar 2013 15:40:20 GMT

This is a feature that is not really generic enough to add to the engine, imho. Also, risks getting jobs stuck because the machine is down. Maybe the future improvements to the jobExecutor could make it easier to plug this kind of approach in without having to rip out the whole jobExecutor as it is now.

Re: Job Acquisition filter

nbromage — Wed, 18 Sep 2013 11:15:56 GMT

Hello Frederik,

Did the job priority configuration changes make it into the 5.13 release?

nick

Re: Job Acquisition filter

frederikherema1 — Fri, 20 Sep 2013 12:56:04 GMT

No job-priority handling has been added to the engine.