topic Lots of timer tasks, slow lock acquisition times in Alfresco Archive

Lots of timer tasks, slow lock acquisition times

kirkb75 — Mon, 23 Jun 2014 21:29:17 GMT

Id like to know if this approach is feasible and I'll try to keep this first post simple. Requirements call for recurring processes which wait and trigger at configurable intervals (weekly, monthly, yearly, etc). 10M+ active processes exist, with 50-100k becoming candidates any given day, each havin

Re: Lots of timer tasks, slow lock acquisition times

jbarrez — Fri, 04 Jul 2014 09:05:45 GMT

Interesting experiment.

I'm wondering though where is the bulk % of the time being spent? Is it the job executor? Is it the execution of the sub process?

How do you start this process? Is it just one? Do you spawn one for every of your entries?

The numbers do seem high, so Im sure there is optimization possible here.

Re: Lots of timer tasks, slow lock acquisition times

kirkb75 — Wed, 09 Jul 2014 16:45:59 GMT

Hey, thanks for the reply.

The bulk of the time is being spent in the job executor when trying to aquire new jobs, within the queries "selectNextJobsToExecute_mysql" and "selectExclusiveJobsToExecute_mysql". Extracting the actual queries and params out using debugging and running in a MySQL client yields very similar results, give or take a second.

The included sub-process is a no-op that was just added for testing, just a simple log statement. Changing it to a script task and/or removing it yields the same overall result.

For brevity I've simplified the code below, but the processes are started by essentially the following inside of a java class (main and/or embedded in a webapp).

for(int i = 0; i < numProcesses; i++) {
   final Map<String, Object> variableMap = new HashMap<String, Object>();
   variableMap.put("ID", String.valueOf(i));
   variableMap.put("nextDate", isoDateString);

   runtimeService.startProcessInstanceByKey("testProcess", String.valueOf(i), variableMap);
}

Also noticed that when starting up a new node in a cluster and deploying processes, there is a "selectJobsByConfiguration" query which gets executed, which also results in a very long startup time for the process being deployed in a new JVM.

Re: Lots of timer tasks, slow lock acquisition times

jbarrez — Tue, 22 Jul 2014 06:43:49 GMT

Is there some way you could share your test code for this? I would like to see and interprete the results for myself.