topic More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields in Alfresco Archive

More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jkronegg — Fri, 25 Jul 2014 07:30:23 GMT

Hi all,I have a setup with about 50 ProcessEngine‍, each connected to a different datasource (one for each company branch), using H2 file database (for testing).Each ProcessEngine‍ uses about 4 MB of memory, including about 3.6 MB for MyBatis Configuration‍ (determined through heap dump analysis).&l

Re: More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jbarrez — Fri, 25 Jul 2014 16:04:00 GMT

Nice! This is a sweet 'hackaround' which makes my development heart tick 😉

Do we need to copy the fields? Can't we just set the Configuration object using reflection, too? (instead of copying those three fields)

Re: More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jkronegg — Sat, 26 Jul 2014 07:33:34 GMT

Well, the fields are not copied, just referenced. But it's true that instead of referencing those three fields in sharedOrmConfigurationObjects, we could reference the first created ProcessEngine's MyBatis configuration fields. But the ProcessEngines keeps a Map<String,ProcessEngine> as a HashMap which is not ordered, so how could we get the first ProcessEngine ?

Re: More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jbarrez — Wed, 30 Jul 2014 07:32:23 GMT

Well, we can always change the implementation of ProcessEngines, of course.

Maybe the first engine that boots up put its Configuration into some shared fields or map of the ProcessEngines class.

Re: More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jkronegg — Thu, 31 Jul 2014 00:44:26 GMT

In fact we could simply modifying the ProcessEngineConfigurationImpl by adding in initSqlSessionFactory(), after configuration.setEnvironment(environment), the following code:


        Collection<ProcessEngine> processEngines = ProcessEngines.getProcessEngines().values();
        if (!processEngines.isEmpty()) {
         ProcessEngine firstProcessEngine = processEngines.iterator().next();
         if (firstProcessEngine instanceof ProcessEngineImpl) {
          ProcessEngineImpl pe = (ProcessEngineImpl)firstProcessEngine;
          Configuration firstConfiguration = pe.getProcessEngineConfiguration().getSqlSessionFactory().getConfiguration();
          shareConfigurationField(configuration, firstConfiguration, "sqlFragments");
          shareConfigurationField(configuration, firstConfiguration, "mappedStatements");
          shareConfigurationField(configuration, firstConfiguration, "resultMaps");
         }
        }

and the following shareConfigurationField() method:


  /**
   * Share a specific field of the MyBatis Configuration. 
   * @param configuration the current configuration
   * @param sharedObjectFieldName the field to be shared
   */
  private static void shareConfigurationObject(final Configuration configuration, final Configuration firstConfiguration, final String sharedObjectFieldName) {
      Throwable exception = null;
      try {
          // get the field to share 
          Field sharedField = Configuration.class.getDeclaredField(sharedObjectFieldName);
          sharedField.setAccessible(true);
          Object sharedObject = sharedField.get(firstConfiguration);

          // we got a shared object => use it in the configuration
          sharedField.set(configuration, sharedObject);

      } catch (NoSuchFieldException e) {
          exception = e;
      } catch (IllegalArgumentException e) {
          exception = e;
      } catch (IllegalAccessException e) {
          exception = e;
      }
      if (exception != null) {
          LOGGER.log(Level.WARNING, "could not configured ORM with shared " + sharedObjectFieldName + " field; " + exception.toString(), exception);
      }
  }

Re: More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jbarrez — Fri, 22 Aug 2014 07:10:48 GMT

Hi Julien,

I finally came around trying out your stuff … and blogging about it: http://www.jorambarrez.be/blog/2014/08/22/seriously-reducing-memory/

Cool stuff!

Re: More than one ProcessEngine: reducing memory footprint by 10x using MyBatis shared configuration fields

jkronegg — Fri, 22 Aug 2014 10:40:54 GMT

Nice blog post Joram. My true name is Julien Kronegg.