JPA process variables not fully supported?

[chris_joelly]

Hello,

i just figured out that JPA entities are defined as not cachable in JPAEntityVariableType.

I have the following use case where this is a real problem:

i use JSF/activiti-cdi, in the JSF based user task views i use things like:

<customer:detail customer="#{processVariables['order'].customer}" />
‍‍‍

I observed that the entity 'customer', and its related entity 'address' is updated correctly when i modify values in the JSF view and execute

<h:commandButton id="complete" value="#{msg.completeTaskButton}" action="#{businessProcess.completeTask(true)}" />
‍‍‍

Following the user task there are service tasks which works on the process variable 'order'.
But when i execute code like

   @Override
   public void execute(DelegateExecution execution) throws Exception {
      Order order = (Order) execution.getVariable(VariableNames.ORDER);
   }
‍‍‍‍‍‍

i see that the values set on the entities just before in the user task are not there.
The reason is that in VariableInstanceEntity the following happens:

  public Object getValue() {
    if (!type.isCachable() || cachedValue==null) {
      cachedValue = type.getValue(this);
    }
    return cachedValue;
  }
‍‍‍‍‍‍‍‍

In the 'order' referenced by cachedValue i see the modified entity from the user task, but due to the fact that JPAEntityVariableType is not cachable
it is read from the database and all modifications are lost.

Is this the expected behavior for JPA process variables?

If so, how can i solve this issue?

Thanks
Chris

[chris_joelly]

After having the idea that the JPA process variable should be persisted when the user task is completed using the BusinessProcess.completeTask() method i tried to find out why it is not persisted. If it then is re-read from the database it would solve my initial problem. Maybe this is not a good thing from the performance perspective, but from the transactional behavior it is a nice thing for sure, as the user task is completed and might be persisted the process variables should be in sync as well.

Maybe i am wrong, but when the process variable is read when the JSF form is opened using #{processVariables['order'] an EntityManagerSession is created and when the JPA variable is read it calls the EntityManagerFactory to obtain an EntityManager, which is then cached in that EMS.

But when the task is completed another EntityManagerSession is created and it gets the EntityManagerFactory from the application server again. But when the flush() method is hit the EntityManager is not initialized, the EMF is never asked to hand out an EM. So, the flush is not performed and the process variable is not persisted.

What am i doing wrong? Is this use case simply not considered (JPA variable with a user task followed by a service task)

Stack Trace when loading the JPA variable:

EntityManagerSessionImpl.getEntityManager() line: 84
JPAEntityMappings.findEntity(Class<?>, Object) line: 123
JPAEntityMappings.getJPAEntity(String, String) line: 119
JPAEntityVariableType.getValue(ValueFields) line: 77
VariableInstanceEntity.getValue() line: 158
ExecutionEntity(VariableScopeImpl).getVariable(String) line: 93
GetExecutionVariableCmd.execute(CommandContext) line: 60
CommandExecutorImpl.execute(Command<T>) line: 24
CommandContextInterceptor.execute(Command<T>) line: 42
JtaTransactionInterceptor.execute(Command<T>) line: 59
LogInterceptor.execute(Command<T>) line: 33
RuntimeServiceImpl.getVariable(String, String) line: 102
UiMediatedBusinessProcessBean(BusinessProcess).getVariable(String) line: 303
ProcessVariableMap.get(Object) line: 43
MapELResolver.getValue(ELContext, Object, Object) line: 196
DemuxCompositeELResolver._getValue(int, ELResolver[], ELContext, Object, Object) line: 176
DemuxCompositeELResolver.getValue(ELContext, Object, Object) line: 203
AstValue.getValue(EvaluationContext) line: 169
ValueExpressionImpl.getValue(ELContext) line: 189
WeldValueExpression.getValue(ELContext) line: 50
TagValueExpression.getValue(ELContext) line: 109
UIComponentBase$AttributesMap.get(Object) line: 2362
CompositeComponentAttributesELResolver$ExpressionEvalMap.get(Object) line: 345
MapELResolver.getValue(ELContext, Object, Object) line: 196
DemuxCompositeELResolver._getValue(int, ELResolver[], ELContext, Object, Object) line: 176
DemuxCompositeELResolver.getValue(ELContext, Object, Object) line: 203
AstValue.getValue(EvaluationContext) line: 169
ValueExpressionImpl.getValue(ELContext) line: 189
WeldValueExpression.getValue(ELContext) line: 50
ContextualCompositeValueExpression.getValue(ELContext) line: 156
TagValueExpression.getValue(ELContext) line: 109
ComponentStateHelper.eval(Serializable, Object) line: 194
ComponentStateHelper.eval(Serializable) line: 182
HtmlOutputText(UIOutput).getValue() line: 169
TextRenderer(HtmlBasicInputRenderer).getValue(UIComponent) line: 205
TextRenderer(HtmlBasicRenderer).getCurrentValue(FacesContext, UIComponent) line: 355
TextRenderer(HtmlBasicRenderer).encodeEnd(FacesContext, UIComponent) line: 164
HtmlOutputText(UIComponentBase).encodeEnd(FacesContext) line: 875
GridRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 312
GridRenderer.renderRow(FacesContext, UIComponent, UIComponent, ResponseWriter) line: 185
GridRenderer.encodeChildren(FacesContext, UIComponent) line: 129
HtmlPanelGrid(UIComponentBase).encodeChildren(FacesContext) line: 845
GridRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 304
GridRenderer.renderRow(FacesContext, UIComponent, UIComponent, ResponseWriter) line: 185
GridRenderer.encodeChildren(FacesContext, UIComponent) line: 129
HtmlPanelGrid(UIComponentBase).encodeChildren(FacesContext) line: 845
GroupRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 304
GroupRenderer.encodeChildren(FacesContext, UIComponent) line: 105
UIPanel(UIComponentBase).encodeChildren(FacesContext) line: 845
UIPanel(UIComponent).encodeAll(FacesContext) line: 1779
CompositeRenderer.encodeChildren(FacesContext, UIComponent) line: 78
UINamingContainer(UIComponentBase).encodeChildren(FacesContext) line: 845
UINamingContainer(UIComponent).encodeAll(FacesContext) line: 1779
FormRenderer(Renderer).encodeChildren(FacesContext, UIComponent) line: 168
HtmlForm(UIComponentBase).encodeChildren(FacesContext) line: 845
OutputPanelRenderer(CoreRenderer).renderChild(FacesContext, UIComponent) line: 55
OutputPanelRenderer(CoreRenderer).renderChildren(FacesContext, UIComponent) line: 43
OutputPanelRenderer.encodeEnd(FacesContext, UIComponent) line: 44
OutputPanel(UIComponentBase).encodeEnd(FacesContext) line: 875
GroupRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 312
GroupRenderer.encodeChildren(FacesContext, UIComponent) line: 105
HtmlPanelGroup(UIComponentBase).encodeChildren(FacesContext) line: 845
GridRenderer(HtmlBasicRenderer).encodeRecursive(FacesContext, UIComponent) line: 304
GridRenderer.renderRow(FacesContext, UIComponent, UIComponent, ResponseWriter) line: 185
GridRenderer.encodeChildren(FacesContext, UIComponent) line: 129
HtmlPanelGrid(UIComponentBase).encodeChildren(FacesContext) line: 845
HtmlPanelGrid(UIComponent).encodeAll(FacesContext) line: 1779
UIOutput(UIComponent).encodeAll(FacesContext) line: 1782
UIViewRoot(UIComponent).encodeAll(FacesContext) line: 1782

Stack Trace when completing the user task and flushing the JPA variable:

EntityManagerSessionImpl.flush() line: 63
JPAEntityVariableType.setValue(Object, ValueFields) line: 61
VariableInstanceEntity.setValue(Object) line: 164
ExecutionEntity(VariableScopeImpl).setVariableInstanceValue(Object, VariableInstanceEntity) line: 196
ExecutionEntity(VariableScopeImpl).setVariableLocal(String, Object) line: 189
ExecutionEntity(VariableScopeImpl).setVariable(String, Object) line: 167
ExecutionEntity(VariableScopeImpl).setVariables(Map<String,Object>) line: 264
TaskEntity.setExecutionVariables(Map<String,Object>) line: 360
CompleteTaskCmd.execute(CommandContext) line: 54
CompleteTaskCmd.execute(CommandContext) line: 28
CommandExecutorImpl.execute(Command<T>) line: 24
CommandContextInterceptor.execute(Command<T>) line: 42
JtaTransactionInterceptor.execute(Command<T>) line: 59
LogInterceptor.execute(Command<T>) line: 33
TaskServiceImpl.complete(String, Map<String,Object>) line: 148
UiMediatedBusinessProcessBean(BusinessProcess).completeTask() line: 263
UiMediatedBusinessProcessBean.completeTask() line: 53
UiMediatedBusinessProcessBean(BusinessProcess).completeTask(boolean) line: 274
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object…) line: 597
AstValue.invoke(EvaluationContext, Class[], Object[]) line: 262
MethodExpressionImpl.invoke(ELContext, Object[]) line: 278
WeldMethodExpression(ForwardingMethodExpression).invoke(ELContext, Object[]) line: 39
WeldMethodExpression.invoke(ELContext, Object[]) line: 50
TagMethodExpression.invoke(ELContext, Object[]) line: 105
MethodBindingMethodExpressionAdapter.invoke(FacesContext, Object[]) line: 88
ActionListenerImpl.processAction(ActionEvent) line: 102
HtmlCommandButton(UICommand).broadcast(FacesEvent) line: 315

So, simply spoken, with this behavior and the fact that JPA variables are not cachable we will not be able to use JPA variables in a way like i do?
Using them via CDI on JSF with user tasks and service tasks?

Chris

[frederikherema1]

Can you show how you configure your engine (the XML), especially the part where the JPA-stuff is configured

[chris_joelly]

Hello,

here is the cfg:


<jee:jndi-lookup id="dataSource" jndi-name="java:/datasources/appDS" />

<!– use transaction manager from the application server –>
<jee:jndi-lookup id="transactionManager" jndi-name="java:jboss/TransactionManager" />
<jee:jndi-lookup id="entityManagerFactory" jndi-name="persistence/appEMF" resource-ref="false" />

<!– integrate Activiti with CDI and JTA –>
<bean id="processEngineConfiguration"
  class="org.activiti.cdi.CdiJtaProcessEngineConfiguration">

  <!– config for JTA managed persistence –>
  <property name="dataSource" ref="dataSource" />
  <property name="transactionManager" ref="transactionManager"/>
  <property name="transactionsExternallyManaged" value="true" />
 
  <property name="jpaEntityManagerFactory" ref="entityManagerFactory" />
  <property name="jpaHandleTransaction" value="false" />
  <property name="jpaCloseEntityManager" value="false" />

  <!– upgrade Activiti model if needed –>
  <property name="databaseSchemaUpdate" value="true" />
  <property name="databaseType" value="mysql" />

  <!– enable timers –>
  <property name="jobExecutorActivate" value="true" />
 
  <!– log full details to the history tables –>
  <property name="history" value="full" />
</bean>


[chris_joelly]

During debugging the whole thing i discovered that the GetExecutionVariableCmd is executed when i open the user task with a JSF form and i access the JPA entity process variable. There a EntityManagerSession is created from the session factory and put into the command context. In that session an entity manager is used to load the entity. After the GetExecutionVariableCmd is finished with its execution the session is flushed and finally closed. In fact the whole command context is discarded.

When i then modify the entity through the JSF form and execute a completeTask which ends in the CompleteTaskCommand, a new command context is build for it. There a new EntityManagerSession is created using the session factory and of course there is no entity manager available. It would only be available when it would have been stored there using the call to getEntityManager(), which unfortunately only happens when JPAEntityMappings.findEntity() would have been called. So, i added this call to EntityManagerSessionImpl.flush(), but it did not help. Well, this for sure has its reason that with that entity manager no entity is associated… i think a merge would be needed so the entity gets flushed using that newly created entity manager.

So… finally i think regardless what i set in the activiti.cfg.xml, the whole thing with JPA entities does not work when a new EntityManagerSession is created for every command…

[frederikherema1]


<property name="jpaEntityManagerFactory" ref="entityManagerFactory" />
      <property name="jpaHandleTransaction" value="false" />
      <property name="jpaCloseEntityManager" value="false" />

A lot depends on your environment? You tell the session NOT to flush, nor close the entityManager. You're correct about the command context being discarded every time an API-call is done, this is intended behavior. There is no easy way of "keeping the same EntityManager" in between API-calls, because activiti is not aware of this off course (can take a long time between API-call, or never called again etc).

Why is there no EntityManager available when the task is completed? The EntityManagerFactory is responsible for creating a new EntityManager when requested. This should have access to the previously stored JPA-entities, no?

[chris_joelly]

Hello,

thank u for your quick response.

If i do not tell the session to flush then why it does so? In fact it flushes even after the entity is read via JpaEntityMapping… And when the CommandContext is closed all session managers are considered and a flush is executed on them. (CommandContext.close() -> closeSessions() -> EntityManagerSessionImpl.flush())

  public void flush() {
    if (entityManager != null && (!handleTransactions || isTransactionActive()) ) {
      try {
        entityManager.flush();
The reason why there is no EntityManager available when the task is completed is that no one is created using the then active EntityManagerSession.
It just tries to use the EntityManager which is held in the EntityManagerSessionImpl instance. But, as u can see in the snippet posted above, getEntityManager() is not called in flush(). The only one which is calling getEntityManager() on EntityManagerSession is JpaEntityMappings when the entity is read when the user task is started (through JPAEntityVariableType.getValue() which calls JPAEntityMappings.getJPAEntity(), and findEntity()). But that session is discarded as mentioned earlier together with its cached EntityManager.

Furthermore i think a em.merge() and em.persist() would be needed somewhere before the flush when the entity is stored to the JPAEntityVariableType?

public void setValue(Object value, ValueFields valueFields) {
    EntityManagerSession entityManagerSession = Context
      .getCommandContext()
      .getSession(EntityManagerSession.class);
Why not get an EntityManager there and perform an em.merge(value)? Then the EntityManagerSession would have an EM for the flush() operation and the JPA entity would be attached to the persistence context? Or do we need to implement the merge and persist logic ourself, and if so, how can we align the containers EntityManager with the one Activiti will use? Activiti will, if it would do it, get an EntityManager using the configured EntityManagerFactory. But for sure it will not get the same EntityManger which is injected to lets say service tasks which resolve to EJBs…

And if i read the code, what ever i would configure with jpaHandleTransaction or jpaCloseEntityManager, it would not make any difference in this case, as the EntityManager is null in my case when the task is completed.

I tried to understand why my rather simple use case which is promoted in the manual and here on the forum and some blog postings is not working, and during some debugging sessions it seems that the code does not support this, at least my, use case.

For a quick try i just added these four lines of code to the JPAEntityVariableType.setValue() method (from 5.10-SNAPSHOT):

  public void setValue(Object value, ValueFields valueFields) {
    EntityManagerSession entityManagerSession = Context
      .getCommandContext()
      .getSession(EntityManagerSession.class);
    if (entityManagerSession == null) {
      throw new ActivitiException("Cannot set JPA variable: " + EntityManagerSession.class + " not configured");
    } else {
     // get EM and merge entity to be flushed afterwards
     EntityManager entityManager = entityManagerSession.getEntityManager();
     if (entityManager != null) {
      entityManager.merge(value);
     }
      // Before we set the value we must flush all pending changes from the entitymanager
      // If we don't do this, in some cases the primary key will not yet be set in the object
      // which will cause exceptions down the road.
      entityManagerSession.flush();
    }
it works nicely for me.

So, what do think, are these two lines of code valid there? Or are there other side effects expected?

Thanks
Chris

[frederikherema1]

Your approach seems valid, up to the point where you "flush" the entity-manager on each "set" of a variable. The entity-manager should only be flushed at the end of the command-context-chain (if all goes well).

So instead of getting the entity-manager directly in the setValue(), rather you should initialize the JPAEntityManagerSession the way the reading of JPA-variables does. This way, IF a variable is set, the entity manager is initialized in the session and will be not-null when session is flushed, resulting in the desired behavior.

Could you try this approach and perhaps attach the patch, so we can introduce this in the acidity code-base?

[chris_joelly]

hm…

i dont understand what u mean with "up to the point where you "flush" the entity-manager on each "set" of a variable".
This is the usual behavior of the code on 5.10-SNAPSHOT, i just added the following four lines, as stated in my previous post:

       // get EM and merge entity to be flushed afterwards
       EntityManager entityManager = entityManagerSession.getEntityManager();
       if (entityManager != null) {
          entityManager.merge(value);
       }
AFAIK a em.merge() does not perform a flush?
The call to entityManagerSession.flush() was already there …

But i will investigate on your proposal…

[frederikherema1]

Sorry, I was mistaken. You're right about the fact that the flush is already there…

No I truly get what is going on and what your use case is. The activiti-variable is only responsible for actually storing the reference to the entity, not handling potential changes in the entity itself. Therefor, the entity isn't merged with the entity-manager, because we're only interested in the ID and classname. In case of a jpa-entity created in the SAME entity-manager, the flush() before getting the ID is done to ensure all JPA-entities have an ID set.

So in your case, where you compete the task with a JPA-entity, activiti assumes you have already saved (has an ID) OR will be assigned an ID if the same entity-manager has been used. You should, before passing on the JPA-entity to activiti, merge it yourself. This is not a responsibility of activiti imho.