Thanks for taking the time to give feedback.
For the why contextReusePossible is set to false. Actually, when running the application (attached to initial post) which starts 10k instances, I systematically reach a point where roughly around 9500 instances terminates, but the engine blocks, the threads are waiting except one or two. I suspected a deadlock, which is usually the case. However, I was surprised to see bunches of commits and update processes set idle and active on the database side, not able proceed due to deadlocks (server status view). In contrast, on the application side only have one or two thread still in runnable state.
I started looking into the code where the commits are done. In the CommandContextInterceptor class, the call to context.close() does the call to commit. However, I found out that if the contextReusePossible is true the commit is not done " if (!contextReused) { context.close();}". It seems to me that explains the bunch of idle commit process on the database side. I wanted the commits to be done as soon as the job is executed. Consequently, switching the contextReusePossible to false by default does the job for me and the my application managed to terminate all processes without blocking of deadlocks.
What, I understand from you that setting contextReusePossible to false by default breaks service calls, you mean developer provided service tasks, right? If this is the case, I can conclude that if an applicaiton having no service tasks, it is will behave correctly right? Or you mean that it breaks Activiti services? In this case, I am eager to know your input on why my application succeeded to terminate all process instances.
Concerning retries. I am wondering whether a retried job gets executed the number of times it is retried. By getting executed, I mean the execute method of the developer DelegateExecution implementation is invoked with every job retry. If this is the case, it might create side-effects, right? Consider the case when a job sends an email, it will sent the email the number of times the job is getting retried right? I understand that, such a situation is tolerable in the case of transactional data change as the transaction is rolled back, nevertheless it requires careful job design to mitigate side-effects.
When I read the code for ExecuteAsyncRunnable, I understand that a job is retried, if an OptimisticLockException is thrown or the developer provide implementation of the DelegateExecution throws and an exception itself right? But when I run my application, I find that the DelegateExecution is executed with every retrial. There is something I do not understand here. Can you please enlighten me on that.
For SQL change "select … for update nowait", it is available with all major databases: Oracle, SQL server, Postgres. In case a lock cannot be obtained immediately, it will raise an error and return right after without blocking which is very interesting to use in the case of Activiti job locking for execution (less resource consumption, more CPU cycles, more interestingly reduces deadlocks). BTW, I noticed huge performance gains 2 to 3 times improvement, when I applied my changes and running the same application attached to the initial post.
Thanks!