topic Re: JobExecutor - Thread management in Alfresco Archive

JobExecutor - Thread management

slash117 — Wed, 04 Sep 2013 14:00:27 GMT

hi everyone,I'm having the following problem(due to the JobExecutor of Activiti) when launching load tests on my application :the java process's memory increses constantly until it saturates (then i get an OutOfMemory with the java process > 4 Go and the Java process is killed by the OS).in the J

Re: JobExecutor - Thread management

jbarrez — Tue, 10 Sep 2013 15:40:55 GMT

1. The default job executor has these setting

<code>
protected int queueSize = 3;
protected int corePoolSize = 3;
private int maxPoolSize = 10;

protected BlockingQueue<Runnable> threadPoolQueue;
protected ThreadPoolExecutor threadPoolExecutor;
</code>

So it's very weird that you are seeing this behavior, as it should throttle itself when it reaches the 10 threads.
So just to confirm: if you got, say, 100 async tasks, you see 100 threads being created?

2. Yes - the whole JobExecutor infrastructure is pluggable

Re: JobExecutor - Thread management

slash117 — Wed, 11 Sep 2013 11:20:04 GMT

hello,
its true that the number of threads doesn't exceed 10 (= MaxPoolSize), but the fact that CorePoolSize < MaxPoolSize means that every time no more async tasks are executed, the number of threads in the pool decreases from 10 to 3. This causes a phenomenon of thread liberation that apparently causes saturation the native memory of the java process(native memory = total process memory - heap memory - permGen memory).
So instead of re-using the threads previously created, the application keeps creating and liberating threads upon the need of execution tasks.

Regards,
Gilbert E.H.

Re: JobExecutor - Thread management

jbarrez — Fri, 13 Sep 2013 09:34:26 GMT

But wouldn't that be the wanted behaviour? I mean, if the load is low, you don't want to keep those 10 threads spinning up?

Probably a better solution would be to have an 'idle period' on them before they are put back into the thread pool?

Re: JobExecutor - Thread management

slash117 — Fri, 13 Sep 2013 15:33:07 GMT

hello Joram,
i think both solutions are good (constant ThreadPool Size or idle period) or at least better than a variable threadPoolSize without a idle period. I chose to implement the constant threadPool size (CoreSize = MaxSize = 10).

But now i have another problem : If all goes as supposed , now i should have 10 threads from the launching until the shutdown of my application. But unfortunately this is not the case. In fact every time activiti throws an exception(functional error or ActivitiLockingException ), the thread is killed and replaced by another one.
At which level do you think i should catch and manage these exceptions to avoid having this behavior.
I tried to use a catch bloc for Exception in the executeJobs method (JobExecutor) but it didn't work. The thread is killed before…

<code>
public void executeJobs(List<String> jobIds) {
    try {
      threadPoolExecutor.execute(new ExecuteJobsRunnable(this, jobIds));
    } catch (RejectedExecutionException ree) {
      rejectedJobsHandler.jobsRejected(this, jobIds);
    } catch (Exception e) {
        log.error("An error occurred while executing the jobs : " + jobIds.toString(), e);
    }
}
</code>

Regards,
Gilbert E.H.

Re: JobExecutor - Thread management

slash117 — Mon, 16 Sep 2013 08:27:21 GMT

hello,
i corrected the problem by adding a catch bloc in the ExecutableJobsRunnable class thus catching the exception thrown before the ThreadPoolExecutor catchs it.

<code>
public void run() {
        final JobExecutorContext jobExecutorContext = new JobExecutorContext();
        final List<String> currentProcessorJobQueue = jobExecutorContext.getCurrentProcessorJobQueue();
        final CommandExecutor commandExecutor = jobExecutor.getCommandExecutor();

        currentProcessorJobQueue.addAll(jobIds);

        Context.setJobExecutorContext(jobExecutorContext);
        try {
            while (!currentProcessorJobQueue.isEmpty()) {
                commandExecutor.execute(new ExecuteJobsCmd(currentProcessorJobQueue.remove(0)));
            }
        } catch (Exception e) {
            log.error("An error occurred while executing the jobs : " + jobIds.toString(), e);
        } finally {
            Context.removeJobExecutorContext();
        }
    }
</code>

Regards,
Gilbert E.H.