Using CDI from JobExecutor

[naag]

Hi all,

we're looking for ways on how we can make use of CDI services from activities executed by the JobExecutor in JBoss AS 7. In http://forums.activiti.org/en/viewtopic.php?f=6&t=4167 Daniel suggested to write a JCA connector and use @MessageDriven beans. I've never done any JCA connectors before and also I cannot find a lot of documentation about it (maybe looking in the wrong place…). Also we would like to stick to a "pure" JBoss AS 7, so no fox platform ce.

Does anybody have an idea what could be a solution? I was thinking along the lines of having a thread pool provided by JBoss, and then using that instead of the default Java ThreadPool that Activiti uses. But I'm not even sure if that makes sense?

By the way, we're mainly looking to make @Inject work. Of course @RequestScoped and so on won't work.

Thanks for all ideas 🙂
Peter

[meyerd]

Nonono, no JMS at all.

JCA allows you to define your own endpoint interfaces. Our interface looks like this:


/**
* Interface to be implemented by a MessageDriven bean handling the execution of
* a job.
*
* @author Daniel Meyer
*
*/
public interface JobExecutionHandler {

  public void executeJob(String jobId, CommandExecutor commandExecutor);

}
Then we have one (1!) MDB deployed as part of the platform.

So what our JCA job executor does is,

• it uses a thread pool provided by the application server (JCA Work Manager = standardized & portable across vendors)


• the thread pool is used to schedule acquisitions (you can have multiple acquisitions, one acquisition serving one or more engines)


• the thread pool is used to execute jobs.


• executor threads invoke the MDB implementing the above shown interface in a *SYNCHRONOUS* way. (This behaves exactly as the activiti job executor with full support for exclusive jobs. I implemented exclusive Jobs and asynchronous continuations in activiti and wrote the documentation on it, so I should know, right?)


• the MDB runs the default activiti org.activiti.engine.impl.cmd.ExecuteJobsCmd


• our platform handles the "context switch" form the platform application into your process application transparently (this is an orthogonal concern).


• in your process you just write <serviceTask activiti:delegateExpression="${nameOfMyCDIBean}" /> and we make sure the thread calling your bean is a legal container thread being decorated with the correct context, behaving exactly as if some EJB inside your application would invoke that bean.


• our job executor implements the same interface as the activiti job executor (looks the same).

The problems this solves:

• Resources can be configured in the application server, even @ runtime (you can add more threads at runtime)


• An MDB is a legal entry point to the Application server, meaning that your thread will be decorated with the right context by the application server. This includes security, the request context for your @RequestScoped CDI beans, Naming …


• If you have multiple process engines (some of our clients use 50+ for multi tenancy you can still have a singe, shared thread pool for these process engines. If you use the default activiti job executor you end up with something like 500 threads and 50 threads polling the database, your DBA will kill you)


• you can implement advanced strategies for load balancing, scheduling etc. This is something we will be working on heavily in the months to come.

See:

• https://app.camunda.com/confluence/display/foxUserGuide/Advanced+Job+Executor


• https://app.camunda.com/confluence/display/foxUserGuide/Configuration+of+the+fox+platform#Configurat...


• https://app.camunda.com/confluence/display/foxUserGuide/Install+fox+platform+on+Glassfish+3.1#Instal...

[dguggi]

thank you daniel, for your detailed explanation!

this solutions sound like a really comprehensive solution (especially the "your dba will kill you" argument is convincing  )

daniel

[ronald_van_kuij]

The argument is convincing, but…..

The current jobexecutor can also be improved to do a lot less polling… Honestly, the current implementation is suboptimal. In a poc I changed it once and got 500% more throughput if lots of async calls are used.

[dguggi]

hi!

what is exactly suboptimal in activiti's current implementation - we plan to use the jobexecutor with lots of async-continuations, so I'd be interested how to improve throughput.
btw. are there any plans to optimize the jobexecutor referring to your poc?

ty, daniel

[ronald_van_kuij]

The jobexecutor polls the database for selecting jobs every second. Often (certainly if you have only one jobexecutor) the jobs will be pinned to this specific jobexecutor. Inserting the job is 1 query per job,  selecting jobs is one query and pinning the jobs is 1 query. If you assume the server that wrote the job will be the one executing it, you can 'pin' it immediately, reducing the results of the 'select jobs' statement and thus processing  and remove one 'update' statement per job.

All this, and some smaller optimizations, resulted in a much higher troughput, at least in the case of not to complex async services. If the processing time of the services themselves is significantly higher than the database actions, your millage will be a lot less.

Keep in mind that the jobexcutor was initially meant for timed things that you needed to process in the future. So in that respect, the implementation is not bad, not at all. It is just suboptimal for pure async things.

I do not know if I still have the code for this and if so, it will need to be cleaned up since it was also a trial to use JCA but that unfortunately stopped due to ending of the internal project.

[mahavirj]

can you describe how to 'pin' the job immediately?

[bernd_ruecker]

Hi Ronald.

I think even the current open source implementation is a bit better than what you describe. And there is an optimization to give yourself a hint internally if a job is created on your note, no query needed. And especially in fox we even have improved the approach and we have a pretty good throughput 🙂

Cheers Bernd

[ronald_van_kuij]

The opensource version of the Fox JobExecutor? Cool… Will definately have a look then. We had some additional improvements to that helped troughput but I do not recall the details. It was never finished though…

Thanks though for mentioning. I read good things about Fox and since it was almost a year ago I gave it a try, I will do that again soon.

Cheers,

Ronald

[bernd_ruecker]

The coolest stuff is only in EE (but as EE customer you get all the sources). Sorry for that, but we have to keep some enterprise stuff to customers only, but that helps to keep the business model healthy and makes our open source engagement possible (currently we are developing with 5 people full time on Activiti/fox!).

[ronald_van_kuij]

I know, don't blame you, not at all…