Database Exceptions in Unittest

[billdoor]

Hello,

I am currently evaluating Activiti for a new project and keep running into seemingly random, database related exceptions like
ProcessInstance was updated by another transaction concurrently,
Deadlock detected,
Variable was concurrently updated in another Procesinstance

in Unittests.

My Process is quite simple:

<process id="randomCalculator" name="Random Calculator" isExecutable="true">
      <startEvent id="theStart" name="Start"></startEvent>

      <serviceTask id="generateRandomTask" name="generate Random Task"
         activiti:async="true" activiti:class="com.company.processes.ServiceTaskImpl"></serviceTask>

      <userTask id="defineRules" name="Rule definition"
         activiti:assignee="kermit">
         <extensionElements>
            <activiti:formProperty id="min" required="true"
               type="long" />
            <activiti:formProperty id="max" type="long"
               required="true" />
         </extensionElements>
      </userTask>

      <parallelGateway id="valuesAndBoundariesPresent" name="Parallel Gateway"></parallelGateway>

      <serviceTask id="CalculateTask" name="Calculate Task"
         activiti:async="true" activiti:class="com.company.processes.CalculateTaskImpl">
      </serviceTask>

      <endEvent id="theEnd"></endEvent>

      <!– FLOW –>
      <sequenceFlow id="startGeneratingValues" sourceRef="theStart"
         targetRef="generateRandomTask"></sequenceFlow>
      <sequenceFlow id="startGettingUserInput" sourceRef="theStart"
         targetRef="defineRules"></sequenceFlow>
      <sequenceFlow id="UserInputRecieved" sourceRef="defineRules"
         targetRef="valuesAndBoundariesPresent"></sequenceFlow>
      <sequenceFlow id="RandomValuesRecieved" sourceRef="generateRandomTask"
         targetRef="valuesAndBoundariesPresent"></sequenceFlow>
      <sequenceFlow id="startCalculation" sourceRef="valuesAndBoundariesPresent"
         targetRef="CalculateTask"></sequenceFlow>
      <sequenceFlow id="finish" sourceRef="CalculateTask"
         targetRef="theEnd"></sequenceFlow>
   </process>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

the ServiceTask generates two random numbers
the user then has to submit 2 values in a form

then the flow is buffered with a paralell gateway to ensure both randoms and userinput are present as procesvariables.

the calculatetask then checks if both random values are in between the user provides values (min, max) and if they are, it adds the two randoms and prints the result into a file.

my activiti.cfg.xml is used and reads this:

<bean id="processEngineConfiguration"
      class="org.activiti.engine.impl.cfg.StandaloneProcessEngineConfiguration">
      <property name="jdbcUrl" value="jdbc:h2:mem:activiti;DB_CLOSE_DELAY=1000" />
      <property name="jdbcDriver" value="org.h2.Driver" />
      <property name="jdbcUsername" value="sa" />
      <property name="jdbcPassword" value="" />
      <property name="databaseSchemaUpdate" value="drop-create" /> <!– also tried true and false –>
      <property name="jobExecutorActivate" value="true" />
   </bean>
‍‍‍‍‍‍‍‍‍‍‍

my unittest is basically the provided template:

public class ProcessTest_randomCalculator {

   @Rule
   public ActivitiRule activitiRule = new ActivitiRule();
   

   @Test
   @Deployment (resources={"bpmn/randomCalculator.bpmn20.xml"})
   public void startProcess() throws Exception {
      RuntimeService runtimeService = activitiRule.getRuntimeService();
      ProcessInstance processInstance = runtimeService.startProcessInstanceByKey("randomCalculator");
      assertTrue(!processInstance.isEnded() && !processInstance.isSuspended());
      assertNotNull(processInstance.getId());

      List<Task> taskList = activitiRule.getTaskService().createTaskQuery().taskAssignee("kermit").list();
      assertTrue(taskList.size()==1);

      Map<String, String> params = new HashMap<String, String>();
      params.put("min", "-100");
      params.put("max", "100");
      activitiRule.getFormService().submitTaskFormData(taskList.get(0).getId(), params);
      
   }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

For what i gather, the problem is that the test crashed in some executions and kept processinstances in the database that clutter up some ids, and lead to random problems, even sometimes working fine. Sometimes the engine stops logging at
"
shutting down Job Executor
processing resource randomCalculator.bpmn20.xml
"
and does nothing, neither execute the process, nor log exceptions, errors or warnings.

Is there a way to tell the engine to completely drop the database when using the ActivitiRule class in tests? Or is there any other configuration i messed up and cannot see?

Thanks in advance for help!

[martin_grofcik]

Hi,

I would use 2 parallel gateways. One to fork execution and the second one (already in the process definition) to join the execution.
(could you attach your jUnit test project too http://forums.activiti.org/content/sticky-how-write-unit-test)
Regards
Martin

[billdoor]

Update:

it could be related to this: http://forums.activiti.org/content/parallel-gateway-and-asynchronous-continuations

Update:

I played a litte with async and sync behaivior:
Important: in every setup it worked at least occaisonally, randomly distributed 1/20 something sometimes two in a row, sometimes 30 times not.

1)
<code>
start
async-randomTask
sync-userTask
gateway
async-calculateTask
end
</code>
results: most of the time "optimistic locking exception", deadlock-detected. updated in concurrent transaction, calculateTask not started

2)
<code>
start
async-randomTask
sync-userTask
gateway
sync-calculateTask
end
</code>
results: 8/10 succeed, 2/10 deadlock,  updated in concurrent transaction

3, 4)
<code>
start
async-randomTask (3) Thread.sleep() for 5 seconds)
sync-userTask (4) Thread.sleep() for 5 seconds  on Unittest before submitting form data)
gateway
async-calculateTask
end
</code>
results 2)&3) = see 1), no change

5)
<code>
start
async-randomTask (Thread.sleep() for 5 seconds)
sync-userTask
gateway
sync-calculateTask
end
</code>
results = NullPointer on calculateTask: execution.getvariable("setBySyncFlow") returns null. HOW?

6)
<code>
start
async-randomTask
sync-userTask  (Thread.sleep() for 5 seconds  on Unittest before submitting form data)
gateway
sync-calculateTask
end
</code>
results = 15/15 succeed.

2,5 and 6 are key to the problem i think.

2 and 6 show that, if the task after the Join is synchrone and the synchrone flow reaches the gateway LAST, everything is fine.
This explains why 6 is all successes, the async task will never take 5 seconds. in case of 2 the input is almost all the time slower than the async flow, but in 2/10 cases the scheduling favors the main thread and the concurrency issue occurs.

I don't see an explanation for 5). How can the sync task after the gateway not access the variables set by the sync flow before the gateway? In addition, how does this relate in any way to the asnyc task taking longer to execute?

If the flow after the gateway is async however, it does not matter who comes first. the only thing i could think of could be, that the main thread considers the process done because there is nothing synchrone left and tries to update the processState to "done" while the async flow wants to start the next task?

With the gateway as Fork suggestion:


7)
<code>
start
gateway
async-randomTask (Thread.sleep() for 5 seconds after setVariable("values") does not matter because of transactions)
sync-userTask
gateway
sync-calculateTask
end
</code>
results = NullPointer on calculateTask: execution.getvariable("setBySyncTask") returns null. (like 5))

😎
<code>
start
gateway
async-randomTask
sync-userTask
gateway
sync-calculateTask
end
</code>
results = mainly like 1) but seems to work significantly more often BUT: in calculateTask execution.getVariable("setByasyncFlow") returns null occaisonally. This never happend before in any setup.


I hope the data helps anyone figuring out what is going on. I think 5) might actually be a bug?


P.S.
My Unittest, the engine.close() is, well, i just left it there, it does not affect the problem at all.

<code>
public class ProcessTest_randomCalculator {

@Rule
public ActivitiRule engine = new ActivitiRule();

@Test
@Deployment (resources={"bpmn/randomCalculator.bpmn20.xml"})
public void startProcess() throws Exception {
  ProcessEngine engine = ProcessEngines.getDefaultProcessEngine();
 
  RuntimeService runtimeService = engine.getRuntimeService();

  ProcessInstance processInstance = runtimeService.startProcessInstanceByKey("randomCalculator");
  assertNotNull(processInstance.getId());

  List<Task> taskList = engine.getTaskService().createTaskQuery().taskAssignee("kermit").list();
  assertTrue(taskList.size()==1);
  Map<String, String> params = new HashMap<String, String>();
  params.put("min", "-1000");
  params.put("max", "1000");
 
  engine.getFormService().submitTaskFormData(taskList.get(0).getId(), params);
  engine.close();
}
}
</code>


randomTask-code:

<code>
public class ServiceTaskImpl implements JavaDelegate  {

public void execute(DelegateExecution execution) throws Exception {
  Random rnd  = new Random();

  long baseline = System.currentTimeMillis();
  execution.setVariable("started", baseline);
  System.err.println("SERVICETASK START");
  rnd.setSeed(System.currentTimeMillis());
  int i = rnd.nextInt() % 100;
  int j = rnd.nextInt() % 100;

  SerializableValuePair values = new SerializableValuePair(i,j);
  execution.setVariable("values", values);
  Thread.sleep(5000);
  System.err.println("SERVICETASK ENDED: " + (System.currentTimeMillis()-baseline) + " millis");
}
}

</code>


calculateTask-code
<code>
public class CalculateTaskImpl implements JavaDelegate  {
  final static Logger logger = LoggerFactory.getLogger("TEST");


public void execute(DelegateExecution execution) throws Exception {
  long baseline = (Long) execution.getVariable("started");
 
  System.err.println("CALCULATION STARTED: " + (System.currentTimeMillis()-baseline) + " millis");
  SerializableValuePair values = (SerializableValuePair) execution.getVariable("values");
  System.err.println("VALUES: " + values.getA() + " + " + values.getB());
  System.err.println("MAX: " + execution.getVariable("max"));
  long max = (Long) execution.getVariable("max");
  long min = (Long) execution.getVariable("min");

  int result = values.getA() + values.getB();
  System.err.println("VARIABLES RETRIEVED: " + min + ", " + max);
 
  if(values.getA()>max || values.getB() > max || values.getA()<min || values.getB() < min) {
   throw new Exception("Values out of bounds.");
  }
 
  File f = new File("C:\\Users\\A538237\\output.txt");
  PrintWriter w = new PrintWriter(f);
  w.println(new Date());
  w.println(min);
  w.println(max);
  w.println("RESULT " + result);
  w.flush();
  w.close();
  System.err.println("CALCULATION ENDED: " + (System.currentTimeMillis()-baseline) + " millis");
 
}
}
</code>

the type SerializableValuePair is just two wrapped longs with getter, setters and implements serializable, it is not the porblem