topic Re: Using CDI from JobExecutor in Alfresco Archive

Using CDI from JobExecutor

naag — Thu, 19 Jul 2012 10:11:55 GMT

Hi all,we're looking for ways on how we can make use of CDI services from activities executed by the JobExecutor in JBoss AS 7. In http://forums.activiti.org/en/viewtopic.php?f=6&t=4167 Daniel suggested to write a JCA connector and use @MessageDriven beans. I've never done any JCA connectors bef

Re: Using CDI from JobExecutor

frederikherema1 — Thu, 19 Jul 2012 10:23:35 GMT

Using a JBOSS-provided pool would definitely make sense.. The job-executor can be subclassed to run the acquired jobs in that pool, rather than in a default java-threadpool, like the DefaultJobExecutor uses.

Re: Using CDI from JobExecutor

naag — Thu, 19 Jul 2012 11:43:12 GMT

So you think that CDI services would then be available? I'm not too familiar yet with the inner workings of CDI and documentation is a little bit sparse 😞

Re: Using CDI from JobExecutor

frederikherema1 — Thu, 19 Jul 2012 12:50:15 GMT

My knowledge of CDI isn't that great rather. Not sure if "being in the right thead-context" would give access to the beans. Let alone that annotation-resolving would occur. That's more something for a CDI-expert I'm afraid as I only know the basics

Re: Using CDI from JobExecutor

naag — Thu, 19 Jul 2012 13:42:14 GMT

OK, I will try to figure it out with the CDI / JCA spec. There must be something somewhere…! Also from Daniel's post in the thread I posted above, it seems that fox platform supports this use case. So somehow, it must be possible. I try to keep this updated here.

Re: Using CDI from JobExecutor

naag — Fri, 20 Jul 2012 07:30:25 GMT

Short update: implementing a custom JobExecutor turns out to be quite easy, and we can send JMS messages from it to a @MessageDriven bean. That bean has CDI services enabled and will execute the job via managementService.executeJob(). Some code:


import javax.ejb.ActivationConfigProperty;
import javax.ejb.MessageDriven;
import javax.inject.Inject;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageListener;
import javax.jms.TextMessage;

import org.activiti.engine.ProcessEngine;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@MessageDriven(activationConfig = {
  @ActivationConfigProperty(propertyName = "destination", propertyValue = "java:/jms/queue/activitiJobExecutor"),
  @ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
  @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue") })
public class JobExecutorBean implements MessageListener {

 protected static Logger log = LoggerFactory.getLogger(JobExecutorBean.class);

 @Inject
 protected ProcessEngine processEngine;

 @Override
 public void onMessage(Message message) {
  String jobId = getJobIdFromMessage(message);

  if (jobId != null) {
   processEngine.getManagementService().executeJob(jobId);
  }
 }

 protected String getJobIdFromMessage(Message message) {
  if (message instanceof TextMessage) {
   try {
    TextMessage textMessage = (TextMessage) message;
    log.debug("Recevied message over JMS: '{}'", textMessage.getText());
    return textMessage.getText();
   } catch (JMSException e) {
    log.error("Error occured while reading JMS TextMessage", e);
    return null;
   }
  }

  return null;
 }
}

The custom JobExecutor basically just connects to the JMS queue obtained via JNDI and posts the Job IDs to that queue in it's executeJobs() method. Basically the MDB replaces the class ExecutorJobsRunnable which DefaultJobExecutor uses to run jobs in it's thread pool. Both ExecutorJobsRunnable and the MDB rely on ExecuteJobsCmd() to do the rest 🙂 Of course this is totally untested, but in case anybody has the same requirement, it might be helpful…

Now let's take a serious look at fox platform ce where this is supposed to be all pretty 🙂

Re: Using CDI from JobExecutor

ronald_van_kuij — Fri, 20 Jul 2012 07:54:11 GMT

Does this also work for timers? And auto acknowledge? Isn't that dangerous in case of failures?

Re: Using CDI from JobExecutor

naag — Fri, 20 Jul 2012 07:58:28 GMT

I just tested it with timers, it works.

Of course you're right, it's pretty untested at the moment. For example JMS will, in our current configuration, try to deliver the message up to 10 times in case of a failure. By acknowledging manually and setting up JMS properly, I think the issues can be avoided. This was just a quick test to see if things work together in general.

So yes, there are still some kinks and also it's still not 100% Java EE compatible because the JobExecutor still uses a Thread for AcquireJobsRunnable.

Re: Using CDI from JobExecutor

bernd_ruecker — Wed, 01 Aug 2012 16:28:57 GMT

Hi Naag.

It is actually not that different from what we do in the fox platform enterprise edition. But there are quite some nuts and bults to figure out and it might even depend on the JBoss version you are using (see https://app.camunda.com/confluence/display/foxUserGuide/Install+fox+platform+on+JBoss+AS+7#InstallfoxplatformonJBossAS7-SupportedJBossASversions for various bugs we are facing). We basically do the whole stuff in a Java EE 6 compliant way (JCA indeed), so we have the stuff ready for Glassfish and the rest as well. But unfortunately the EE parts of that story are not open source (but we deliver all sources to our customers), but let me know if it is interesting for you anyway.

Cheers
Bernd

Re: Using CDI from JobExecutor

dguggi — Thu, 02 Aug 2012 22:26:25 GMT

I'd be interested how your implementation behaves in respect to exclusive-jobs:

does your custom job-executor (or maybe the activiti base class you extend - didn't check source yet) provide the (http://activiti.org/userguide/#exclusiveJobs) feature?

i think that even if the your custom-job does exclusiveJobs, I'm afraid you may be run into problems, because you send async (jms) messages per jobId (means your impl actually does not support exclusive-jobs feature) - the application-server will typically process messages in parallel.

so if I'm correct this could be solved with
a) ensure sequential processing of messages (which is actually bad imho)
b) use advanced messaging-features (e.g. activemq's messagedestinations etc.)

did you address/solve this issue using jms/mdbs?

besides that, the solution feels somehow hacky/(way too) complex to me (imho too much overhead)
jobexecutor with threadpool => asynchronous jms messaging => mdb

daniel

Re: Using CDI from JobExecutor

meyerd — Fri, 03 Aug 2012 06:18:43 GMT

Nonono, no JMS at all.

JCA allows you to define your own endpoint interfaces. Our interface looks like this:


/**
 * Interface to be implemented by a MessageDriven bean handling the execution of
 * a job.
 * 
 * @author Daniel Meyer
 * 
 */
public interface JobExecutionHandler {

  public void executeJob(String jobId, CommandExecutor commandExecutor);

}

Then we have one (1!) MDB deployed as part of the platform.

So what our JCA job executor does is,

it uses a thread pool provided by the application server (JCA Work Manager = standardized & portable across vendors)

the thread pool is used to schedule acquisitions (you can have multiple acquisitions, one acquisition serving one or more engines)

the thread pool is used to execute jobs.

executor threads invoke the MDB implementing the above shown interface in a *SYNCHRONOUS* way. (This behaves exactly as the activiti job executor with full support for exclusive jobs. I implemented exclusive Jobs and asynchronous continuations in activiti and wrote the documentation on it, so I should know, right?)

the MDB runs the default activiti org.activiti.engine.impl.cmd.ExecuteJobsCmd

our platform handles the "context switch" form the platform application into your process application transparently (this is an orthogonal concern).

in your process you just write <serviceTask activiti:delegateExpression="${nameOfMyCDIBean}" /> and we make sure the thread calling your bean is a legal container thread being decorated with the correct context, behaving exactly as if some EJB inside your application would invoke that bean.

our job executor implements the same interface as the activiti job executor (looks the same).

The problems this solves:

Resources can be configured in the application server, even @ runtime (you can add more threads at runtime)

An MDB is a legal entry point to the Application server, meaning that your thread will be decorated with the right context by the application server. This includes security, the request context for your @RequestScoped CDI beans, Naming …

If you have multiple process engines (some of our clients use 50+ for multi tenancy you can still have a singe, shared thread pool for these process engines. If you use the default activiti job executor you end up with something like 500 threads and 50 threads polling the database, your DBA will kill you)

you can implement advanced strategies for load balancing, scheduling etc. This is something we will be working on heavily in the months to come.

See:

https://app.camunda.com/confluence/display/foxUserGuide/Advanced+Job+Executor

https://app.camunda.com/confluence/display/foxUserGuide/Configuration+of+the+fox+platform#Configurationofthefoxplatform-JobExecutorandJobAcquisitionConfiguration

https://app.camunda.com/confluence/display/foxUserGuide/Install+fox+platform+on+Glassfish+3.1#InstallfoxplatformonGlassfish31-ConfigurationviaAdministrationConsole

Re: Using CDI from JobExecutor

dguggi — Fri, 03 Aug 2012 08:44:43 GMT

thank you daniel, for your detailed explanation!

this solutions sound like a really comprehensive solution (especially the "your dba will kill you" argument is convincing )

daniel

Re: Using CDI from JobExecutor

ronald_van_kuij — Sat, 04 Aug 2012 19:48:57 GMT

The argument is convincing, but…..

The current jobexecutor can also be improved to do a lot less polling… Honestly, the current implementation is suboptimal. In a poc I changed it once and got 500% more throughput if lots of async calls are used.

Re: Using CDI from JobExecutor

dguggi — Mon, 06 Aug 2012 07:35:50 GMT

hi!

what is exactly suboptimal in activiti's current implementation - we plan to use the jobexecutor with lots of async-continuations, so I'd be interested how to improve throughput.
btw. are there any plans to optimize the jobexecutor referring to your poc?

ty, daniel

Re: Using CDI from JobExecutor

ronald_van_kuij — Mon, 06 Aug 2012 08:58:20 GMT

The jobexecutor polls the database for selecting jobs every second. Often (certainly if you have only one jobexecutor) the jobs will be pinned to this specific jobexecutor. Inserting the job is 1 query per job, selecting jobs is one query and pinning the jobs is 1 query. If you assume the server that wrote the job will be the one executing it, you can 'pin' it immediately, reducing the results of the 'select jobs' statement and thus processing and remove one 'update' statement per job.

All this, and some smaller optimizations, resulted in a much higher troughput, at least in the case of not to complex async services. If the processing time of the services themselves is significantly higher than the database actions, your millage will be a lot less.

Keep in mind that the jobexcutor was initially meant for timed things that you needed to process in the future. So in that respect, the implementation is not bad, not at all. It is just suboptimal for pure async things.

I do not know if I still have the code for this and if so, it will need to be cleaned up since it was also a trial to use JCA but that unfortunately stopped due to ending of the internal project.

Re: Using CDI from JobExecutor

bernd_ruecker — Mon, 06 Aug 2012 19:29:13 GMT

Hi Ronald.

I think even the current open source implementation is a bit better than what you describe. And there is an optimization to give yourself a hint internally if a job is created on your note, no query needed. And especially in fox we even have improved the approach and we have a pretty good throughput 🙂

Cheers Bernd

Re: Using CDI from JobExecutor

ronald_van_kuij — Tue, 07 Aug 2012 00:11:03 GMT

The opensource version of the Fox JobExecutor? Cool… Will definately have a look then. We had some additional improvements to that helped troughput but I do not recall the details. It was never finished though…

Thanks though for mentioning. I read good things about Fox and since it was almost a year ago I gave it a try, I will do that again soon.

Cheers,

Ronald

Re: Using CDI from JobExecutor

bernd_ruecker — Tue, 07 Aug 2012 04:40:29 GMT

The coolest stuff is only in EE (but as EE customer you get all the sources). Sorry for that, but we have to keep some enterprise stuff to customers only, but that helps to keep the business model healthy and makes our open source engagement possible (currently we are developing with 5 people full time on Activiti/fox!).

Re: Using CDI from JobExecutor

ronald_van_kuij — Tue, 07 Aug 2012 06:26:49 GMT

I know, don't blame you, not at all…

Re: Using CDI from JobExecutor

mahavirj — Sun, 03 Aug 2014 15:40:09 GMT

can you describe how to 'pin' the job immediately?