Transaction rollback help in Activiti 5.9 BPM engine using Spring

[wickedelephant]

We are having an issue with Activiti BPM engine not rolling back transactions under all circumstances.

This is an Activiti engine that is tightly embedded into our JPA/Spring application.

Most of the time, such as when a service throws an exception, the Activiti BPM transactions are rolled back correctly with the rest of the application.

However, if the JVM is shut down or killed while Activiti is running a process instance, that process instance ends in a corrupt state and all kinds of bad things happen to it.

Here is my configuration:

    <bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close"
            p:name="jdbc/FDDataSource"
            p:jmxEnabled="true"
            p:driverClassName="${jdbc.driverClassName}" p:url="${jdbc.url}" 
            p:defaultTransactionIsolation="2"
            p:username="${jdbc.username}"
            p:password="${jdbc.password}"
            p:initialSize="25"
            p:minIdle="25"
            p:maxIdle="50"
            p:maxActive="200"
            p:maxWait="15000"
            p:logAbandoned="true"
            p:suspectTimeout="60"
            p:timeBetweenEvictionRunsMillis="30000"
            p:minEvictableIdleTimeMillis="60000"
            p:testOnBorrow="true"
            p:validationQuery="SELECT 1"
            p:validationInterval="15000"
            p:jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer" />

    <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"
            p:entityManagerFactory-ref="entityManagerFactory" />

    <!– JPA EntityManagerFactory –>
    <bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="jpaVendorAdapter">
            <bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
                <property name="showSql" value="${jpa.showSql}" />
                <property name="database" value="${jpa.database}" />
            </bean>
        </property>
        <property name="jpaDialect">
            <bean class="org.springframework.orm.jpa.vendor.HibernateJpaDialect" />
        </property>
        <property name="jpaPropertyMap">
            <map>
                <!– Database Configuration –>
                <entry key="hibernate.dialect" value="org.hibernate.dialect.MySQLInnoDBDialect" />
                <entry key="hibernate.hbm2ddl.auto" value="${hibernate.hbm2ddl.auto}" />
                <entry key="hibernate.format_sql" value="false" />
                <entry key="hibernate.use_sql_comments" value="true" />
                <entry key="hibernate.ejb.metamodel.generation" value="disabled" />
                <entry key="hibernate.default_batch_fetch_size" value="50" />
                <entry key="hibernate.order_updates" value="true" />
                <!– READ_COMMITTED –>
                <entry key="hibernate.connection.isolation" value="2" />
                <!– Cache Configuration –>
                <entry key="hibernate.cache.use_query_cache" value="false" /> 
                <entry key="hibernate.cache.use_second_level_cache" value="false" />
                <!– Transaction Management –>
                <entry key="hibernate.current_session_context_class" value="org.springframework.orm.hibernate3.SpringSessionContext" />
            </map>
        </property>     
        <!– Custom implementation to enrich the PersistenceUnitInfo read from the persistence.xml
             JPA configuration file with the JTA datasource. specifying the JTA datasource directly in
             the Spring configuration file has the advantage that we can use a direct reference to the
             datasource instead of using a JNDI name as requied by the jta-data-source setting in the
             persistence.xml file –>
        <property name="persistenceUnitPostProcessors">
            <bean class="com.fd.service.jta.JtaPersistenceUnitPostProcessor">
                <property name="jtaDataSource" ref="dataSource" />      
            </bean>
        </property> 
        <property name="loadTimeWeaver">
            <bean class="org.springframework.instrument.classloading.ReflectiveLoadTimeWeaver" />
        </property>
    </bean>

    <bean id="sessionFactory" factory-bean="entityManagerFactory" factory-method="getSessionFactory" />

<context:annotation-config transaction-manager="transactionManager" />

    <!–
        Instruct Spring to perform declarative transaction management
        automatically on annotated classes.
    –>
    <tx:annotation-driven mode="aspectj" transaction-manager="transactionManager" />

    <!–
        Instruct Spring to retrieve and apply @AspectJ aspects which are defined
        as beans in this context (such as the CallMonitoringAspect below).
    –>
    <aop:aspectj-autoproxy />

    <!–
        Post-processor to perform exception translation on @Repository classes (from native
        exceptions such as JPA PersistenceExceptions to Spring's DataAccessException hierarchy).
    –>
    <bean class="org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor" />

    <!–
    ~~~~~~~~~~~~~~~~~~  ACTIVITI BPM  ~~~~~~~~~~~~~~~~~~
    –>

    <bean id="activitiProcessEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration">
        <property name="dataSource" ref="dataSource" />
        <property name="transactionManager" ref="transactionManager" />
        <property name="databaseSchemaUpdate" value="false" />
        <property name="jobExecutorActivate" value="false" />
        <!– Do not uncomment or process definitions will auto-deploy upon server startup in prod. –>
        <!– <property name="deploymentResources" value="classpath*:/com/fd/bpm/*.bpmn20.xml" /> –>
        <property name="history" value="audit" />
    </bean>

    <bean id="activitiProcessEngine" class="org.activiti.spring.ProcessEngineFactoryBean">
        <property name="processEngineConfiguration" ref="activitiProcessEngineConfiguration" />
    </bean>

    <bean id="activitiRepositoryService" factory-bean="activitiProcessEngine" factory-method="getRepositoryService" />
    <bean id="activitiRuntimeService" factory-bean="activitiProcessEngine" factory-method="getRuntimeService" />
    <bean id="activitiTaskService" factory-bean="activitiProcessEngine" factory-method="getTaskService" />
    <bean id="activitiHistoryService" factory-bean="activitiProcessEngine" factory-method="getHistoryService" />
    <bean id="activitiManagementService" factory-bean="activitiProcessEngine" factory-method="getManagementService" />
    <bean id="activitiIdentityService" factory-bean="activitiProcessEngine" factory-method="getIdentityService" />
    <bean id="activitiFormService" factory-bean="activitiProcessEngine" factory-method="getFormService" />
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Is there something wrong with my configuration? Or is this an issue with the transaction management code in the Activiti BPM engine?

[jbarrez]

Activiti simply defers to the Spring transactionmanager here.

"However, if the JVM is shut down or killed while Activiti is running a process instance, that process instance ends in a corrupt state and all kinds of bad things happen to it."

Can you explain what you mean with corrupt state? Does something gets committed to the db? Activiti flushes at the end of the transaction, so it seems odd your data gets corrupted if the JVM failed.

Any more logging or so when that happens?

[wickedelephant]

Thank you for your reply. 

Yes, changes are being committed to the database and the tasks in the process can no longer be completed. Sometimes we will end up with more than one concurrently running task instance where there is only one defined in the process flow.

To be honest, I was kind of hoping someone would say something along the lines of "yes, this is a known issue" or "your configuration is not correct" or "transactions don't work with JPA without a JTA transaction manager."

Unfortunately, I do not have the incident isolated with logs or other debugging data because we have only been able to identify it in production and it is very rare.  We have millions of process instances and this has only happened maybe a few hundred times total.  So it would be very difficult to pin point it.  I have not been able to duplicate the issue in our development environment.

[jbarrez]

Hmm then it's indeed looking for a needle in a haystack :s.