Camel context : activiti endpoint lost

[mvincent]

Hi

  We are currently encountering a weird problem on our production environment.

Every day, it seems that the camel context is losing the activiti endpoints as we get the following exception :

Exception or error caught in resource
java.lang.RuntimeException: Activiti endpoint not defined for activiti://process:task
   at org.activiti.camel.CamelBehaviour.getEndpoint(CamelBehaviour.java:59)
   at org.activiti.camel.CamelBehaviour.createEndpoint(CamelBehaviour.java:50)
   at org.activiti.camel.CamelBehaviour.execute(CamelBehaviour.java:41)
   at org.activiti.engine.impl.delegate.ActivityBehaviorInvocation.invoke(ActivityBehaviorInvocation.java:34)
   at org.activiti.engine.impl.delegate.DelegateInvocation.proceed(DelegateInvocation.java:37)
   at org.activiti.engine.impl.delegate.DefaultDelegateInterceptor.handleInvocation(DefaultDelegateInterceptor.java:25)
   at org.activiti.engine.impl.bpmn.behavior.ServiceTaskDelegateExpressionActivityBehavior.execute(ServiceTaskDelegateExpressionActivityBehavior.java:69)
   at org.activiti.engine.impl.pvm.runtime.AtomicOperationActivityExecute.execute(AtomicOperationActivityExecute.java:44)
   at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:87)
   at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:532)
   at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:527)

Once we restart the tomcat everything is going well.
First, I thought it was the "known" problem (http://forums.activiti.org/content/activiti-engine-hangs-every-few-hours-tomcat) about datasource so I switched to BoneCP but still falling.

We are using the activiti engine embedded in a war application exposing a REST API which interact with the activiti endpoints.

Any idea about a possible cause ?
How can I get some more logs to figure out the initial problem ?

Thanks in advance
Mat

[frederikherema1]

Don't immediatly see what can be causing this. Getting more logs, depends on your container/environment. I'm suspecting it's in the camel-code, so perhaps raise the logging-level of the camel-classes…

[mvincent]

Hi Frederik,

sorry for late reply didn't see your post

Still facing the problem and even with log level to DEBUG on Activiti & camel, nothing seems to fail…

We are deploying the application in a Tomcat v7 with jdk1.7.

I join the logs about Activiti engine and camel (start failing @ 09:59:41).
Also, following my process engine configuration.

As far as I can see, the engine seems ok as I can start new process instances even after "failure", but it seems that either on call activity to start my subprocess or one first step on this subprocess with call an activiti endpoint (to request a REST service) something fails and activiti endpoints are not defined anymore.

Engine configuration

<code>
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration">
  <property name="databaseType">
   <utilroperty-path path="sitiDataSource.type" />
  </property>
  <property name="dataSource" ref="sitiDataSource" />
  <property name="transactionManager" ref="transactionManager" />
  <property name="databaseSchemaUpdate" value="true" />
  <property name="deploymentResources" value="classpath*:diagrams/*.bpmn" />
  <property name="history" value="none" />
  <property name="jobExecutorActivate" value="true" />
  <property name="jobExecutor" ref="defaultJobExec" />
  <property name="idGenerator">
   <bean class="org.activiti.engine.impl.persistence.StrongUuidGenerator" />
  </property>
</bean>

<bean id="defaultJobExec" class="org.activiti.engine.impl.jobexecutor.DefaultJobExecutor">
  <property name="corePoolSize" value="20" />
  <property name="queueSize" value="10" />
  <property name="maxPoolSize" value="25" />
</bean>
</code>

Thanks in advance
Mat

[mvincent]

I may figured out the origin of the problem.

In CamelBehavior, when trying to retrieve my activiti endpoint I get the previous exception. After debugging the activiti sources, it appears that camelcontext endpoint cache is full and thus remove some endpoints !
Now, I need to find how to make my activiti endpoint not evicted from cache or externalized in another camel context maybe.

Any suggestion for a workaround/correction ?

Mat

[mvincent]

Hi,

  here is the class containing the workaround I have to implement to fix this problem.

Mat

[trademak]

Hmm strange, do you have a lot of endpoint definitions in your Camel context?

Best regards,

[smirzai]

It look strange, But what I can see in logs is that you are using depricated CamelBehaviour.java  and not the standard CamelBehavior.java

But in your fix, you are extending from the correct one.
It is difficult for me to image that you have millions of endpoints, considering that endpoints are created only once when the component is initialiezed.
How does it run with the correct behavior without fixes ?

[gabrielfrancisc]

I'm having the same issue. It looks like that Camel keeps the dynamic REST urls and the endpoints definition keep growing in runtime.

For instance, I have an endpoint like:
/rs/my-endpoint/{id}

If I try to get:
/rs/my-endpoint/1

and get:
/rs/my-endpoint/2

It is being saved two different endpoint definitions dynamically. Kind of weird.

Well, My question is: how can I define a global CamelBehavior to resolve this problem without the need to set the CamelBehavior in every task?

In the CamelBehavior class, there's the javadoc that says:

<strong>* The chosen implementation should be set within your ProcessEngineConfiguration. To specify the implementation using Spring, include
* the following line in your configuration file as part of the properties for "org.activiti.spring.SpringProcessEngineConfiguration":
*
*   <property name="camelBehaviorClass" value="org.activiti.camel.impl.CamelBehaviorCamelBodyImpl"/>
</strong>
but, I can't find anywhere to define this in the SpringProcessEngineConfiguration class. Does anyone can help?

Thanks in advance!