topic Re: JPA process variables not fully supported? in Alfresco Archive

JPA process variables not fully supported?

chris_joelly — Sat, 28 Jul 2012 10:47:43 GMT

Hello,i just figured out that JPA entities are defined as not cachable in JPAEntityVariableType.I have the following use case where this is a real problem:i use JSF/activiti-cdi, in the JSF based user task views i use things like:<customer:detail customer="#{processVariables['order'].customer}" /

Re: JPA process variables not fully supported?

chris_joelly — Sat, 28 Jul 2012 18:10:09 GMT

After having the idea that the JPA process variable should be persisted when the user task is completed using the BusinessProcess.completeTask() method i tried to find out why it is not persisted. If it then is re-read from the database it would solve my initial problem. Maybe this is not a good thing from the performance perspective, but from the transactional behavior it is a nice thing for sure, as the user task is completed and might be persisted the process variables should be in sync as well.

Maybe i am wrong, but when the process variable is read when the JSF form is opened using #{processVariables['order'] an EntityManagerSession is created and when the JPA variable is read it calls the EntityManagerFactory to obtain an EntityManager, which is then cached in that EMS.

But when the task is completed another EntityManagerSession is created and it gets the EntityManagerFactory from the application server again. But when the flush() method is hit the EntityManager is not initialized, the EMF is never asked to hand out an EM. So, the flush is not performed and the process variable is not persisted.

What am i doing wrong? Is this use case simply not considered (JPA variable with a user task followed by a service task)

Stack Trace when loading the JPA variable:


 EntityManagerSessionImpl.getEntityManager() line: 84 
 JPAEntityMappings.findEntity(Class<?>, Object) line: 123 
 JPAEntityMappings.getJPAEntity(String, String) line: 119 
 JPAEntityVariableType.getValue(ValueFields) line: 77 
 VariableInstanceEntity.getValue() line: 158 
 ExecutionEntity(VariableScopeImpl).getVariable(String) line: 93 
 GetExecutionVariableCmd.execute(CommandContext) line: 60 
 CommandExecutorImpl.execute(Command<T>) line: 24 
 CommandContextInterceptor.execute(Command<T>) line: 42 
 JtaTransactionInterceptor.execute(Command<T>) line: 59 
 LogInterceptor.execute(Command<T>) line: 33 
 RuntimeServiceImpl.getVariable(String, String) line: 102 
 UiMediatedBusinessProcessBean(BusinessProcess).getVariable(String) line: 303 
 ProcessVariableMap.get(Object) line: 43 
 MapELResolver.getValue(ELContext, Object, Object) line: 196 
 DemuxCompositeELResolver._getValue(int, ELResolver[], ELContext, Object, Object) line: 176 
 DemuxCompositeELResolver.getValue(ELContext, Object, Object) line: 203 
 AstValue.getValue(EvaluationContext) line: 169 
 ValueExpressionImpl.getValue(ELContext) line: 189 
 WeldValueExpression.getValue(ELContext) line: 50 
 TagValueExpression.getValue(ELContext) line: 109 
 UIComponentBase$AttributesMap.get(Object) line: 2362 
 CompositeComponentAttributesELResolver$ExpressionEvalMap.get(Object) line: 345 
 MapELResolver.getValue(ELContext, Object, Object) line: 196 
 DemuxCompositeELResolver._getValue(int, ELResolver[], ELContext, Object, Object) line: 176 
 DemuxCompositeELResolver.getValue(ELContext, Object, Object) line: 203 
 AstValue.getValue(EvaluationContext) line: 169 
 ValueExpressionImpl.getValue(ELContext) line: 189 
 WeldValueExpression.getValue(ELContext) line: 50 
 ContextualCompositeValueExpression.getValue(ELContext) line: 156 
 TagValueExpression.getValue(ELContext) line: 109 
 ComponentStateHelper.eval(Serializable, Object) line: 194 
 ComponentStateHelper.eval(Serializable) line: 182 
 HtmlOutputText(UIOutput).getValue() line: 169 
 TextRenderer(HtmlBasicInputRenderer).getValue(UIComponent) line: 205 
 TextRenderer(HtmlBasicRenderer).getCurrentValue(FacesContext, UIComponent) line: 355 
 TextRenderer(HtmlBasicRenderer).encodeEnd(FacesContext, UIComponent) line: 164 
 HtmlOutputText(UIComponentBase).encodeEnd(FacesContext) line: 875 
 GridRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 312 
 GridRenderer.renderRow(FacesContext, UIComponent, UIComponent, ResponseWriter) line: 185 
 GridRenderer.encodeChildren(FacesContext, UIComponent) line: 129 
 HtmlPanelGrid(UIComponentBase).encodeChildren(FacesContext) line: 845 
 GridRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 304 
 GridRenderer.renderRow(FacesContext, UIComponent, UIComponent, ResponseWriter) line: 185 
 GridRenderer.encodeChildren(FacesContext, UIComponent) line: 129 
 HtmlPanelGrid(UIComponentBase).encodeChildren(FacesContext) line: 845 
 GroupRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 304 
 GroupRenderer.encodeChildren(FacesContext, UIComponent) line: 105 
 UIPanel(UIComponentBase).encodeChildren(FacesContext) line: 845 
 UIPanel(UIComponent).encodeAll(FacesContext) line: 1779 
 CompositeRenderer.encodeChildren(FacesContext, UIComponent) line: 78 
 UINamingContainer(UIComponentBase).encodeChildren(FacesContext) line: 845 
 UINamingContainer(UIComponent).encodeAll(FacesContext) line: 1779 
 FormRenderer(Renderer).encodeChildren(FacesContext, UIComponent) line: 168 
 HtmlForm(UIComponentBase).encodeChildren(FacesContext) line: 845 
 OutputPanelRenderer(CoreRenderer).renderChild(FacesContext, UIComponent) line: 55 
 OutputPanelRenderer(CoreRenderer).renderChildren(FacesContext, UIComponent) line: 43 
 OutputPanelRenderer.encodeEnd(FacesContext, UIComponent) line: 44 
 OutputPanel(UIComponentBase).encodeEnd(FacesContext) line: 875 
 GroupRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 312 
 GroupRenderer.encodeChildren(FacesContext, UIComponent) line: 105 
 HtmlPanelGroup(UIComponentBase).encodeChildren(FacesContext) line: 845 
 GridRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 304 
 GridRenderer.renderRow(FacesContext, UIComponent, UIComponent, ResponseWriter) line: 185 
 GridRenderer.encodeChildren(FacesContext, UIComponent) line: 129 
 HtmlPanelGrid(UIComponentBase).encodeChildren(FacesContext) line: 845 
 HtmlPanelGrid(UIComponent).encodeAll(FacesContext) line: 1779 
 UIOutput(UIComponent).encodeAll(FacesContext) line: 1782 
 UIViewRoot(UIComponent).encodeAll(FacesContext) line: 1782

Stack Trace when completing the user task and flushing the JPA variable:


 EntityManagerSessionImpl.flush() line: 63 
 JPAEntityVariableType.setValue(Object, ValueFields) line: 61 
 VariableInstanceEntity.setValue(Object) line: 164 
 ExecutionEntity(VariableScopeImpl).setVariableInstanceValue(Object, VariableInstanceEntity) line: 196 
 ExecutionEntity(VariableScopeImpl).setVariableLocal(String, Object) line: 189 
 ExecutionEntity(VariableScopeImpl).setVariable(String, Object) line: 167 
 ExecutionEntity(VariableScopeImpl).setVariables(Map<String,Object>) line: 264 
 TaskEntity.setExecutionVariables(Map<String,Object>) line: 360 
 CompleteTaskCmd.execute(CommandContext) line: 54 
 CompleteTaskCmd.execute(CommandContext) line: 28 
 CommandExecutorImpl.execute(Command<T>) line: 24 
 CommandContextInterceptor.execute(Command<T>) line: 42 
 JtaTransactionInterceptor.execute(Command<T>) line: 59 
 LogInterceptor.execute(Command<T>) line: 33 
 TaskServiceImpl.complete(String, Map<String,Object>) line: 148 
 UiMediatedBusinessProcessBean(BusinessProcess).completeTask() line: 263 
 UiMediatedBusinessProcessBean.completeTask() line: 53 
 UiMediatedBusinessProcessBean(BusinessProcess).completeTask(boolean) line: 274 
 NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] 
 NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39 
 DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25 
 Method.invoke(Object, Object…) line: 597 
 AstValue.invoke(EvaluationContext, Class[], Object[]) line: 262 
 MethodExpressionImpl.invoke(ELContext, Object[]) line: 278 
 WeldMethodExpression(ForwardingMethodExpression).invoke(ELContext, Object[]) line: 39 
 WeldMethodExpression.invoke(ELContext, Object[]) line: 50 
 TagMethodExpression.invoke(ELContext, Object[]) line: 105 
 MethodBindingMethodExpressionAdapter.invoke(FacesContext, Object[]) line: 88 
 ActionListenerImpl.processAction(ActionEvent) line: 102 
 HtmlCommandButton(UICommand).broadcast(FacesEvent) line: 315

So, simply spoken, with this behavior and the fact that JPA variables are not cachable we will not be able to use JPA variables in a way like i do?
Using them via CDI on JSF with user tasks and service tasks?

Chris

Re: JPA process variables not fully supported?

frederikherema1 — Tue, 31 Jul 2012 10:30:40 GMT

Can you show how you configure your engine (the XML), especially the part where the JPA-stuff is configured

Re: JPA process variables not fully supported?

chris_joelly — Tue, 31 Jul 2012 19:38:14 GMT

Hello,

here is the cfg:


 <jee:jndi-lookup id="dataSource" jndi-name="java:/datasources/appDS" />
 
 <!– use transaction manager from the application server –>
 <jee:jndi-lookup id="transactionManager" jndi-name="java:jboss/TransactionManager" />
 <jee:jndi-lookup id="entityManagerFactory" jndi-name="persistence/appEMF" resource-ref="false" />

 <!– integrate Activiti with CDI and JTA –>
 <bean id="processEngineConfiguration"
  class="org.activiti.cdi.CdiJtaProcessEngineConfiguration">

  <!– config for JTA managed persistence –>
  <property name="dataSource" ref="dataSource" />
  <property name="transactionManager" ref="transactionManager"/>
  <property name="transactionsExternallyManaged" value="true" />
  
  <property name="jpaEntityManagerFactory" ref="entityManagerFactory" />
  <property name="jpaHandleTransaction" value="false" />
  <property name="jpaCloseEntityManager" value="false" />

  <!– upgrade Activiti model if needed –>
  <property name="databaseSchemaUpdate" value="true" />
  <property name="databaseType" value="mysql" />

  <!– enable timers –>
  <property name="jobExecutorActivate" value="true" />
  
  <!– log full details to the history tables –>
  <property name="history" value="full" />
 </bean>

Re: JPA process variables not fully supported?

chris_joelly — Tue, 31 Jul 2012 21:25:21 GMT

During debugging the whole thing i discovered that the GetExecutionVariableCmd is executed when i open the user task with a JSF form and i access the JPA entity process variable. There a EntityManagerSession is created from the session factory and put into the command context. In that session an entity manager is used to load the entity. After the GetExecutionVariableCmd is finished with its execution the session is flushed and finally closed. In fact the whole command context is discarded.

When i then modify the entity through the JSF form and execute a completeTask which ends in the CompleteTaskCommand, a new command context is build for it. There a new EntityManagerSession is created using the session factory and of course there is no entity manager available. It would only be available when it would have been stored there using the call to getEntityManager(), which unfortunately only happens when JPAEntityMappings.findEntity() would have been called. So, i added this call to EntityManagerSessionImpl.flush(), but it did not help. Well, this for sure has its reason that with that entity manager no entity is associated… i think a merge would be needed so the entity gets flushed using that newly created entity manager.

So… finally i think regardless what i set in the activiti.cfg.xml, the whole thing with JPA entities does not work when a new EntityManagerSession is created for every command…

Re: JPA process variables not fully supported?

frederikherema1 — Wed, 01 Aug 2012 06:05:47 GMT


<property name="jpaEntityManagerFactory" ref="entityManagerFactory" />
      <property name="jpaHandleTransaction" value="false" />
      <property name="jpaCloseEntityManager" value="false" />

A lot depends on your environment? You tell the session NOT to flush, nor close the entityManager. You're correct about the command context being discarded every time an API-call is done, this is intended behavior. There is no easy way of "keeping the same EntityManager" in between API-calls, because activiti is not aware of this off course (can take a long time between API-call, or never called again etc).

Why is there no EntityManager available when the task is completed? The EntityManagerFactory is responsible for creating a new EntityManager when requested. This should have access to the previously stored JPA-entities, no?

Re: JPA process variables not fully supported?

chris_joelly — Wed, 01 Aug 2012 20:46:34 GMT

Hello,

thank u for your quick response.

If i do not tell the session to flush then why it does so? In fact it flushes even after the entity is read via JpaEntityMapping… And when the CommandContext is closed all session managers are considered and a flush is executed on them. (CommandContext.close() -> closeSessions() -> EntityManagerSessionImpl.flush())


  public void flush() {
    if (entityManager != null && (!handleTransactions || isTransactionActive()) ) {
      try {
        entityManager.flush();

The reason why there is no EntityManager available when the task is completed is that no one is created using the then active EntityManagerSession.
It just tries to use the EntityManager which is held in the EntityManagerSessionImpl instance. But, as u can see in the snippet posted above, getEntityManager() is not called in flush(). The only one which is calling getEntityManager() on EntityManagerSession is JpaEntityMappings when the entity is read when the user task is started (through JPAEntityVariableType.getValue() which calls JPAEntityMappings.getJPAEntity(), and findEntity()). But that session is discarded as mentioned earlier together with its cached EntityManager.

Furthermore i think a em.merge() and em.persist() would be needed somewhere before the flush when the entity is stored to the JPAEntityVariableType?


public void setValue(Object value, ValueFields valueFields) {
    EntityManagerSession entityManagerSession = Context 
      .getCommandContext()
      .getSession(EntityManagerSession.class);

Why not get an EntityManager there and perform an em.merge(value)? Then the EntityManagerSession would have an EM for the flush() operation and the JPA entity would be attached to the persistence context? Or do we need to implement the merge and persist logic ourself, and if so, how can we align the containers EntityManager with the one Activiti will use? Activiti will, if it would do it, get an EntityManager using the configured EntityManagerFactory. But for sure it will not get the same EntityManger which is injected to lets say service tasks which resolve to EJBs…

And if i read the code, what ever i would configure with jpaHandleTransaction or jpaCloseEntityManager, it would not make any difference in this case, as the EntityManager is null in my case when the task is completed.

I tried to understand why my rather simple use case which is promoted in the manual and here on the forum and some blog postings is not working, and during some debugging sessions it seems that the code does not support this, at least my, use case.

For a quick try i just added these four lines of code to the JPAEntityVariableType.setValue() method (from 5.10-SNAPSHOT):


  public void setValue(Object value, ValueFields valueFields) {
    EntityManagerSession entityManagerSession = Context 
      .getCommandContext()
      .getSession(EntityManagerSession.class);
    if (entityManagerSession == null) {
      throw new ActivitiException("Cannot set JPA variable: " + EntityManagerSession.class + " not configured");
    } else {
     // get EM and merge entity to be flushed afterwards
     EntityManager entityManager = entityManagerSession.getEntityManager();
     if (entityManager != null) {
      entityManager.merge(value);
     }
      // Before we set the value we must flush all pending changes from the entitymanager
      // If we don't do this, in some cases the primary key will not yet be set in the object
      // which will cause exceptions down the road.
      entityManagerSession.flush();
    }

it works nicely for me.

So, what do think, are these two lines of code valid there? Or are there other side effects expected?

Thanks
Chris

Re: JPA process variables not fully supported?

frederikherema1 — Thu, 02 Aug 2012 06:39:32 GMT

Your approach seems valid, up to the point where you "flush" the entity-manager on each "set" of a variable. The entity-manager should only be flushed at the end of the command-context-chain (if all goes well).

So instead of getting the entity-manager directly in the setValue(), rather you should initialize the JPAEntityManagerSession the way the reading of JPA-variables does. This way, IF a variable is set, the entity manager is initialized in the session and will be not-null when session is flushed, resulting in the desired behavior.

Could you try this approach and perhaps attach the patch, so we can introduce this in the acidity code-base?

Re: JPA process variables not fully supported?

chris_joelly — Thu, 02 Aug 2012 11:33:25 GMT

hm…

i dont understand what u mean with "up to the point where you "flush" the entity-manager on each "set" of a variable".
This is the usual behavior of the code on 5.10-SNAPSHOT, i just added the following four lines, as stated in my previous post:


       // get EM and merge entity to be flushed afterwards
       EntityManager entityManager = entityManagerSession.getEntityManager();
       if (entityManager != null) {
          entityManager.merge(value);
       }

AFAIK a em.merge() does not perform a flush?
The call to entityManagerSession.flush() was already there …

But i will investigate on your proposal…

Re: JPA process variables not fully supported?

frederikherema1 — Thu, 02 Aug 2012 11:51:57 GMT

Sorry, I was mistaken. You're right about the fact that the flush is already there…

No I truly get what is going on and what your use case is. The activiti-variable is only responsible for actually storing the reference to the entity, not handling potential changes in the entity itself. Therefor, the entity isn't merged with the entity-manager, because we're only interested in the ID and classname. In case of a jpa-entity created in the SAME entity-manager, the flush() before getting the ID is done to ensure all JPA-entities have an ID set.

So in your case, where you compete the task with a JPA-entity, activiti assumes you have already saved (has an ID) OR will be assigned an ID if the same entity-manager has been used. You should, before passing on the JPA-entity to activiti, merge it yourself. This is not a responsibility of activiti imho.

Re: JPA process variables not fully supported?

chris_joelly — Thu, 02 Aug 2012 14:19:52 GMT

ok.

well, this conclusion matches with my conclusion after i studied the code during my debugging sessions. I was heavily missleaded by the user guide and some blog postings which say that Activiti supports JPA entities as process variables, which it actually does when it comes to reading entities from the database. When it comes to storing them, and for sure people want to modify them during user tasks and service tasks, then Activiti support is not there.

Especially the blog post by Bernd Rücker (http://www.bpm-guide.de/2012/04/04/pageflow-vs-process-flow-and-ui-mediator-pattern/) gives the impression that using activiti-cdi together with JSF and other JEE technologies makes such use cases possible. Only JPA variables are not covered.. Because they dont work out of the box in such an environment when they are modified through JSF and wants to get persisted when BusinessProcess.completeTask() is executed from an JSF command …

Re: JPA process variables not fully supported?

frederikherema1 — Thu, 02 Aug 2012 14:28:25 GMT

Yes, that's true, when working in such environments (and activiti participates in such transitions/contexts) the updates to the variables themselves are reflected as where activiti out of the box, only uses the id's. I didn't grasp the initial problem you had, so tok a while to see

Re: JPA process variables not fully supported?

chris_joelly — Sat, 04 Aug 2012 17:31:17 GMT

I want to come back to the following statements:

Your approach seems valid, up to the point where you "flush" the entity-manager on each "set" of a variable. The entity-manager should only be flushed at the end of the command-context-chain (if all goes well).

What is the command-context-chain? When i complete a user task, this starts a command-context-chain till execution arrives at the next wait state? Or from the start process to some wait state? Well, with the ensure-to-get-the-enities-id-flush this strategy already seems broken. From a performance perspective this might have an influence if many JPA entities are used as process variables and a flush is executed after each setValue. But if something goes bad a roll back is executed and the state of the process will still be valid, in terms of the execution and the data are synchronized?

So instead of getting the entity-manager directly in the setValue(), rather you should initialize the JPAEntityManagerSession the way the reading of JPA-variables does. This way, IF a variable is set, the entity manager is initialized in the session and will be not-null when session is flushed, resulting in the desired behavior.

With JPAEntityManagerSession u mean the EntityManagerSessionImpl? if i got u right this is what i did when adding the em.merge… What about removing the entityManagerSession.flush() from the JPAEntityVariableType.setValue() call? This is only needed if JPA entities are created within Activiti tasks which dont have their identity set? In my case this would never be the case because the processes will always work on already existing entities. But as it has only a minor performance impact i think its not a big deal?

Could you try this approach and perhaps attach the patch, so we can introduce this in the acidity code-base?

Well, that 4 lines of codes … if Activiti is designed the way that it does not touch JPA entities it might introduce side effects on other users which expect Activiti to not change their JPA entities? Maybe it is possible to introduce it with another jpaXXX config parameter for activiti.cfg.xml which defaults to the actual behavior?

Re: JPA process variables not fully supported?

chris_joelly — Sun, 05 Aug 2012 16:35:14 GMT

Today i tried to create an JPA entity from within an service task which is executed via an process which is triggered by a start timer event and set this newly created entity as process variable. The entity is just instantiated with new and no PersistenceContext was involved. only setters are called to set its member variables.

I did not expect that it would work, but the things i observed was somehow strange

The summary is:

the entity was persisted two times, with two different ids, but only the second one was stored as process variable.

Now the long story

I think the 'problem' is in the method VariableScopeImpl.createVariableLocal() where the entity is stored 'twice' to the variableInstance, first with the VariableInstanceEntity.createAndInsert() call where the entity is modified during the flush and merge and then with the setVariableInstanceValue(value, variableInstance), which still use the unmodified entity because the merge and flush operations are not propagated back to the original entity.

I just added a value = variableInstance.getValue() just before the setVariableInstanceValue(value, variableInstance) and re-read the entity from the persistence context. And now the second entity is not created and everything looks fine. At least for my use cases.

So, finally i replaced the entityManagerSession.flush() with an em.merge() in JPAEntityVariableType.setValue() and added the above mentioned line to VariableScopeImpl.createVariableLocal() to make the persistence of existing and new JPA entities work with Activiti.

What do u think? Would this introduce side effects?

One downside i could already see when i run the Activiti unit tests:

Here is one of the JPA entities used in the JPA test cases, and how they look before and after the merge:

before:
org.activiti.standalone.jpa.FieldAccessJPAEntity@4260ab

after:
org.apache.openjpa.enhance.org$activiti$standalone$jpa$FieldAccessJPAEntity$pcsubclass@df88d2

and because the @Entity annotation is not declared @Inherited it seems that the OpenJPA enhancement removes the Entity annotation of the JPA entities when they are merged to a persistence contet…

So, the detection if an object is an JPA entity breaks now when using OpenJPA as JPA provider when a merge is performed. It seems that the unit tests never run with entities which belonged to an persistence context?

some related stuff:
http://greensopinion.blogspot.co.at/2009/08/how-i-lost-my-annotations-jpa-hibernate.html
http://struberg.wordpress.com/2012/01/08/jpa-enhancement-done-right/