Dynamic JavaDelegates

[mmaker1234]

Hello Activiti developers, community,

We have a requirement to provide the ability to load new and update workflow process definitions and their corresponding JavaDelegates during the application run. And before you to send me to the forum search let me note that I already read the following:

1. Deploying delegate classes
2. Multiple Classloaders per Engine
3. Support Deployment-specific Classloaders
4. Add business archive classloading for delegation classes
5. Deploying current version of classes with process definition
6. ReflectUtil.loadClass() and custom classloader
7. JavaDelegate instances should not be cached
8. Setting a context classloader when calling client code
9. Activiti Classloading
10. Activiti 5.13 User Guide

Unfortunately I did not found a solution to our problem - there are lot of questions and approaches but I can not see a solution. Did I miss or overlook some solution already provided in the forum?

Our environment is the following:

• Enterprise application (EAR) running in WebLogic application server;
• Activiti Engine 5.13 provided as a library in the EAR;
• Activiti Engine configuration file (activiti.cfg.xml) provided in the EAR;
• We do not use Spring;
• Due to security reasons we have to expose the minimum possible web interfaces, therefore we can use neither the engine REST API nor the Aciviti Explorer;
• We have a separate, command line application to upload the process definitions to the database. It actually invokes a separate, standalone Activiti Engine instance connected to the same database that the enterprise application uses;

What we need to achieve is:

• To be able to add or update a process definition at any time during the enterprise application run;
• To be able to change/update the JavaDelegates at any time during the enterprise application run;
• The changes in the JavaDelegates must apply immediately. Preferably immediate after the (re-) deployment (i.e. for the already existing process instances) but latest with the next created process instance;
• We are not allowed to restart the application (as this is the only way to restart the Activiti Engine) to apply the changes.

We mainly elaborated on Daniel's tips. Unfortunately

• "deactivate the deployment cache (implement a subclass which does not cache anything)" is not possible because the

  DeploymentManagerresolveProcessDefinition(ProcessDefinitionEntity)‍

  heavily relays on cache. We tried with an analog of

  org.activiti.standalone.deploy.CustomDeploymentCache‍

  but in our test with a single definition the changes in the JavaDelegates were not reflected (until engine restart). The test result draws this solution as at least unreliable - we can not guarantee that instances of other process definitions will be served in order to force the cache update;
• "use proxy delegates (a delegate which loads the "actual" delegate)" is not quite applicable as we should loose or at least have big headaches with the transfer of the fields of the JavaDelegate;
• "do not use java delegates (but lookup objects the lifecycle of which is managed by some other container like EJB/CDI, OSGi, Spring, … )" To be honest we do not understand this proposal quite good. We tried with DelegateExpression but it seems there is no big difference with the JavaDelegate - instead to provide the delegate class name in the process definition, one should provide that class name in the engine configuration (which is read only once - at the engine start)
• "You have to garbage collect the classloader that loaded a class / create a new instance of your classloader" How to garbage collect the class loader when

  ReflectUtil.getCustomClassLoader()‍

  returns the result of

  processEngineConfiguration.getClassLoader()‍

  which always provides the same object?!
• In regard to JavaDelegate instances should not be cached: "I propose that we do not cache DelegateInstances in ClassDelegate", Resolved, Fixed in 5.9. I do not observe this fix in 5.13 - the javadoc for

  DefaultDeploymentCache‍

  states "keep everything in memory, unless a limit is set".

Currently we did the following to satisfy the requirements:

• Created a custom class loader;
• Provided the class loader in the Activiti Engine configuration;
• Provided for each process definition a deployment package (.jar) containing all JavaDelegates referred by the process definition. These packages are deployed in the application server as separate applications;
• Forced

  ClassDelegate.execute(ActivityExecution)‍

  and

  ClassDelegate.signal(ActivityExecution)‍

  to always call

  getActivityBehaviorInstance(execution)‍

  ;

What we achieved is:

• We can add or update the process definition at any time and it applies with the next process instantiation;
• We can add (deploy) the corresponding JavaDelegates and they apply with their reference by the process instance;

What we still miss is the update of the JavaDelegates to take place immediately and not only after the Activiti Engine restart.

Now the question is (phew, at last!): (again) How to achieve dynamic JavaDelegates update (in an application server environment)?

By the way, anyone to see a feasible or promising approach we miss?

Thank you in advance for your efforts to answer,
Monique

[felipe1]

Monique, have you looked the OSGI approach? It's described in chapter 9 from the book Activiti in Action. I do not use it, personally, but it seems to be a good fit in your scenario.

I'm interested in this subject, so I'll keep an eye in this post. Hopefully someone with more experience will post a more appropriate answer.

[trademak]

May I ask which application server you are using?
We are thinking about providing more support around this for JBoss 7.
Right now the best approach would be to use EJB's for you service logic. Then you can invoke that EJB from the Java delegate. If you keep the interface flexible you should be able to replace the EJB component with a new version.

Best regards,

[mmaker1234]

Hello Tijs,

About the beginning of my original post I already stated that: "Our environment is the following:
    Enterprise application (EAR) running in WebLogic application server;"

To be honest I do not catch with your idea with the "adapter" JavaDelegate. My concern is that I can not make the JavaDelegate flexible enough: what would happen if I need to add another field (delegate parameter) in the process definition? Then I would need (again) to refresh the JavaDelegate class in run time. Or there is other way to get the fields from within the JavaDelegate in order to provide them to the EJB? Nevertheless I suppose that the call to the JavaDelegate will fail with missing setter/field when the new version of the process definition is instantiated (and the JavaDelegate "missed/failed" to update). Do I miss something in the JavaDelegate invocation mechanism?

Another option could be to change the process definition to parametrize the JavaDelegate using variables instead of fields. Then to make the delegate to "know" the EJB to call and flexible enough to look-up a setter for each process instance variable (while not failing on missing setters). This approach means:

1. To mandate the use of statefull EJBs, which are much difficult to maintain;
2. The whole mechanism of JavaDelegate parameters definition becomes hidden
3. as one need to read all the process to reveal what value a variable might have in a certain activity;
 
4. the EJB setters must be carefully reviewed on each process definition change
   and error prone - at least there will be no error if some parameter (variable) is missing a setter (due to the flexibility described above)

Cheers,
Monique

[mmaker1234]

OK, we have something that looks like a solution.

It seems that adding the pgadecki's change to our "Forced <code>ClassDelegate.execute(ActivityExecution)</code> and <code>ClassDelegate.signal(ActivityExecution)</code> to always call <code>getActivityBehaviorInstance(execution)</code>" we got something that satisfies our needs.

We tested each change separately and any of them alone does not work for us.

Now the question is: Are these changes critical to the work of the Activiti Engine? Subquestions:

• What is the reason the variable <code>activityBehaviorInstance</code> to be global in <code>ClassDelegate</class>? It is shared only between <code>execute</code> and <code>signal</code> methods. AFAIR there is no activity to be simultaneously executable and signallable but the implementation provides some kind of a hidden cache. Nevertheless, once the code is written in this manner probably I'm missing something …;
• Why it is so critical to use <code>clazz = Class.forName(className, true, classLoader)</code> in <code>ReflectUtilloadClass(String)</code> in the case a (custom) ClassLoader is available?

P.S. Here are the changes in <code>ReflectUtil</code>:
<code>  public static Class<?> loadClass(String className) {
   Class<?> clazz = null;
   ClassLoader classLoader = getCustomClassLoader();
  
   // First exception in chain of classloaders will be used as cause when no class is found in any of them
   Throwable throwable = null;
  
   if(classLoader != null) {
     try {
       LOG.trace("Trying to load class with custom classloader: {}", className);
//       clazz = Class.forName(className, true, classLoader);
       clazz = classLoader.loadClass( className );
     } catch(Throwable t) {
       throwable = t;
     }
   }
    …
</code>and <code>ClassDelegate</code>:
<code>  // Activity Behavior
  public void execute(ActivityExecution execution) throws Exception {
//    if (activityBehaviorInstance == null) {
//      activityBehaviorInstance = getActivityBehaviorInstance(execution);
//    }
    ActivityBehavior activityBehaviorInstance = getActivityBehaviorInstance(execution);
    try {
      activityBehaviorInstance.execute(execution);
    } catch (BpmnError error) {
      ErrorPropagation.propagateError(error, execution);
    }
  }

  // Signallable activity behavior
  public void signal(ActivityExecution execution, String signalName, Object signalData) throws Exception {
//    if (activityBehaviorInstance == null) {
//      activityBehaviorInstance = getActivityBehaviorInstance(execution);
//    }
    ActivityBehavior activityBehaviorInstance = getActivityBehaviorInstance(execution);
   
    if (activityBehaviorInstance instanceof SignallableActivityBehavior) {
      ((SignallableActivityBehavior) activityBehaviorInstance).signal(execution, signalName, signalData);
    } else {
      throw new ActivitiException("signal() can only be called on a " + SignallableActivityBehavior.class.getName() + " instance");
    }
  }</code>

[trademak]

Hi,

Why not implement the routing logic in a stateless EJB as well? So from the java delegate class you just pass along execution info + the process variables to a fixed EJB. Then in this EJB you invoke the EJB you need. This makes the process definition independent from the EJB logic. Yes this means you will have to work with variables instead of field declarations.

About your changes. The activityBehaviorInstance is global because it's instantiated only once and reused when the same service task is executed again. This is for performance reasons.
In don't understand your comment about ReflectUtil. Class.forName(className, true, classLoader) uses the custom classloader to load the class, so why would you want to change that?

Best regards,

[mmaker1234]

Hello Tijs,

Thank you for your answer.

Let me start from the Class.forName() behavior. Our investigation (debug, actually, including the Java runtime) revealed that the Class.forName(String, boolean, ClassLoader) calls the private native method Class.forName0(String, boolean, ClassLoader). On the first call to it, forName0() really reaches the (our custom) ClassLoader findClass() method. On the subsequent calls for the same class name and class loader, though, forName0() directly returns, most probably using some internal caches, the class instance thus never reaching again the findClass() in the class loader. At least this is the behavior of JRE 1.7.40 on Windows. I can not claim how JRE behaves on Linux.

As an opposite, classLoader.loadClass(String) always reaches the ClassLoader findClass(). Obviously that class caching mechanism is not implemented/activated in loadClass() call sequence or internal implementation.

Now lets get back to the JavaDelegates. I personally think that your solution:
1. Is overcomplicated for our case. Nevertheless it might be just the right solution for other Activiti Engine users;
2. Trades the process definition clarity for speed.

As our usage of the Activiti Engine (currently) does not require high throughput we will sacrifice the engine performance for flexibility and process definition clarity by going with the above solution.

Paweł Gadecki's approach ( http://forums.activiti.org/comment/8802#comment-8802 ) is also tempting but I'm afraid it requires much more changes.

Regards,
Monique

[trademak]

Hi Monique,

And that's why we use the Class.forName code in the ReflectUtil class, because by default we want to cache the class instance for reasons I previously explained. But your use case is valid of course. So let's think about a possible solution that could be implemented in the Activiti Engine module.

To start we need to get clear requirements about what's needed. Let me give a first try at that and you can change it to your needs:

The Service task class instance should be recreated at every execution of the service task activity. Right now the instance is cached and this can only be done by changing the Activiti Engine code.

We could add an activiti:singleton="false" attribute. When this is added no class caching is done and a new instance of the Class is created at every execution. Would that fit your requirements?

Best regards,

[mmaker1234]

Hello Tijs,

Yes, our current vision is that we need "The Service task class instance should be recreated at every execution of the service task activity" and I think that an attribute will work well for us.

Thank you for your receptivity! 🙂
Monique

P.S. Once we started a discussion and you opened to changes in the Activiti Engine regarding the JavaDelegate dynamics, what about Paweł Gadecki's idea of providing a separate class loader for each deployment? Or it is too risky for the Engine stability? Is this approach much different from the use case with a custom class loader (provided in the engine configuration) in the perspective of the Engine security and stability?

Let me elaborate a little:

• Our approach allows separate deployment of the JavaDelegates and the process definition:
• The JavaDelegate distribution is separate from the process definition distribution;
• The JavaDelegate deployment is handled by the environment (usually an application server) and not the Activiti Engine;
• A JavaDelegate replacement immediately affects the running instances of all versions of all process definitions
• Any change in a JavaDelegate must be made with the thought of compatibility with older versions of the deployed process definition. (We are aware of this and are ready to take care for the backwards compatibility but others might not notice this effect)
• Requires the new JavaDelegate version to be deployed on each node of a cluster configuration.


• Paweł's approach ties the JavaDelegates implementation to the process definition version, thus encapsulating all process definition supplements (JavaDelegates) in a dedicated execution environment
• The JavaDelegate is bundled in the distribution package of the process definition;
• The JavaDelegate distribution is handled by the Activiti Engine database. An access to it is provided via the Engine deployment management API;
• A change in a JavaDelegate affects exactly one version of the process definition;
• To apply the changes in a JavaDelegate the process definition have to be re-deployed;
• The changes will take effect only with the new instances of the process definition;
• Removal/bypassing of the Activiti Engine caching mechanism is not required;
• Provides better scalability (in a cluster) due to the common deployments repository

Whether the above features of the both approaches are pros or cons each Activiti Engine user should decide for himself.

[rolf]

Hello!

Pawel's approach is exactly what I need!
I want to package the process definition and delegates in a jar-file and deploy it.
Each version of the process definition sould have its own version of delegates.

Is there any sample code?
How do I get the Execution Context in the class loader?

Thanks for your help!