Slow on update using WS

marcosvasousa — Tue, 04 Dec 2012 12:19:42 GMT

I found that the line is very slow final UpdateResult[] updateResults = repositoryServiceSoapBindingStub.update(cml);It takes at first call 13 seconds and subsequentes call takes 5 seconds.I put MYSQL logging query and no slow query founded. Also, I indexed the repository. No solution give to me goo

Re: Slow on update using WS

marcosvasousa — Wed, 05 Dec 2012 12:00:10 GMT

Please, can someone give to me any clue?

Re: Slow on update using WS

marcosvasousa — Mon, 07 Jan 2013 14:09:41 GMT

I tried alfresco version 4 enterprise trial and founded that updating an document with history/versions tooks to long also!

See numbers:
Alfresco 4 Enterprise Trial:
659 ms while updating an document with 39 history
1583 ms while updating an document with 185 history

Alfresco 3 Community:
722 ms while updating an document with 39 history
5289 ms while updating an document with 185 history

Is this normal numbers? Alfresco looks to be slow while more documents are being created!

Re: Slow on update using WS

mrogers — Mon, 07 Jan 2013 16:02:33 GMT

Well at least alfresco 4 is faster.

Yes there seems to be an issue there. Not sure why document history is having that effect. However your system still seems to
be running slowly. First thing to check is the amount of RAM configured.

Also I suggest you don't continue to use a deprecated API.

Re: Slow on update using WS

marcosvasousa — Mon, 07 Jan 2013 16:33:24 GMT

Yes, looks like Alfresco 4 is very fast!

I read the http://chemistry.apache.org/java/0.4.0/maven/apidocs/org/apache/chemistry/opencmis/client/api/Document.html and found "boolean refresh". If use false Alfresco 4 is fast:

Alfresco 4 Enterprise Trial (Linux 64 bits):
329 ms while updating an document with 39 history
635 ms while updating an document with 185 history
…
752 ms while updating an document with 250 history
…
With 1000 documents without refresh, the times decrease from 7 seconds to 3.8 seconds.

We are under JVM memory: "server -Xss1M -Xms3G -Xmx6G -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:NewSize=1G -XX:MaxPermSize=256M -XX:CMSInitiatingOccupancyFraction=80 -"
so I don't think RAM is problem!!!
The machine has 8GB RAM.
Our Linux machine monitoring didn't indicates problems related to memory or disk!

Let me post my new code, so I really believe no longer using deprecated API in Alfresco 4!

Re: Slow on update using WS

marcosvasousa — Mon, 07 Jan 2013 16:43:48 GMT

Below found complete code used.
All my test showed to me that Alfresco will be slow while updating an document with version. Strange why document history do this effect!
What do you advice to me? Remember we are using Alfresco 3 Community in production.
Any additional information that can I provide to you?

package com.xpto.alfresco4;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.text.MessageFormat;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.chemistry.opencmis.client.api.CmisObject;
import org.apache.chemistry.opencmis.client.api.Document;
import org.apache.chemistry.opencmis.client.api.Folder;
import org.apache.chemistry.opencmis.client.api.Session;
import org.apache.chemistry.opencmis.client.api.SessionFactory;
import org.apache.chemistry.opencmis.client.runtime.SessionFactoryImpl;
import org.apache.chemistry.opencmis.commons.PropertyIds;
import org.apache.chemistry.opencmis.commons.SessionParameter;
import org.apache.chemistry.opencmis.commons.data.ContentStream;
import org.apache.chemistry.opencmis.commons.enums.BindingType;
import org.apache.chemistry.opencmis.commons.enums.VersioningState;
import org.apache.chemistry.opencmis.commons.exceptions.CmisObjectNotFoundException;
import org.apache.commons.lang.StringUtils;

public class CreateDocumentUsingCMIS {

   private static final Pattern PATTERN = Pattern.compile("\\{([A-Za-z_]+)\\}");

   private static final String rootPath = "/app:company_home/cm:XptoFE";

   /**
    * @param args
    */
   public static void main(String[] args) throws Exception {
      for (int i = 0; i < 1000; i++) {
         System.out.println(i);
         elementosInformativo();
      }
   }

   private static void elementosInformativo() throws Exception {
      Map<String, String> filteredMetadataProps = new HashMap<String, String>();
      filteredMetadataProps.put("tipo_documento", "elementos_informativos_entidade");
      filteredMetadataProps.put("entidade", "182179");// 182178

      InputStream dataStream = new FileInputStream(new File("C:\\Doc.pdf"));

      long ini = System.currentTimeMillis();
      createDocument("/cm:Entidades/cm:{entidade}/cm:ElementosInformativosEntidade", "elementos_informativos_entidade", "application/pdf", filteredMetadataProps, true, dataStream, true);
      System.out.println("Tempo Elementos Informativo: " + (System.currentTimeMillis() - ini) + " milisegundos.");
   }

   public static void createDocument(final String path, final String type, final String contentType, final Map<String, String> metadata, final boolean supportsVersions,
         final InputStream dataStream, final boolean override) throws Exception {
      // Translate path.
      final String beforeEncodePath = rootPath + translate(path, new HashMap<String, String>(metadata));
      final String documentName = DocumentBrokerUtils.getNodeName(beforeEncodePath);
      final String parentPath = DocumentBrokerUtils.getParentPath(beforeEncodePath);
      final String cleanParentPath = cleanPath(parentPath);

      Session session = getAlfrescoSession();
      CmisObject cmisObject = getCmisObject(session, cleanPath(beforeEncodePath));

      Map<String, Object> properties = new HashMap<String, Object>();
      properties.put(PropertyIds.OBJECT_TYPE_ID, "D:imp:documento");
      properties.put(PropertyIds.NAME, documentName);
      for (final Entry<String, String> entry : metadata.entrySet()) {
         properties.put("imp:" + entry.getKey(), entry.getValue());
      }

      System.out.println(properties);

      byte[] bytes = DocumentBrokerUtils.readFile(dataStream);
      ContentStream contentStream = session.getObjectFactory().createContentStream(documentName, (long) (bytes.length), contentType, new ByteArrayInputStream(bytes));

      if (cmisObject == null) {
         // Criar o ficheiro com versionamento
         System.out.println(MessageFormat.format("Creating new document: {0}", beforeEncodePath));

         // Build and validate path.
         buildAndValidateFullPath(session, cleanParentPath);

         CmisObject cmisObjectFolder = session.getObjectByPath(cleanParentPath);
         Folder folder = (Folder) cmisObjectFolder;
         folder.createDocument(properties, contentStream, VersioningState.MAJOR);
      } else {
         // Actualizar o ficheiro com versionamento
         System.out.println(MessageFormat.format("Overriding existing document: {0}", beforeEncodePath));

         Document document = (Document) cmisObject;
         document.setContentStream(contentStream, true, false);
         cmisObject.updateProperties(properties, false);
      }
   }

   private static Session getAlfrescoSession() {
      // 1. Authentication Session
      Map<String, String> parameter = new HashMap<String, String>();

      // Set the user credentials
      parameter.put(SessionParameter.USER, "admin");
      parameter.put(SessionParameter.PASSWORD, "PASSWORD");

      // Specify the connection settings
      parameter.put(SessionParameter.ATOMPUB_URL, "http://HOST:PORT/alfresco/service/cmis");
      parameter.put(SessionParameter.BINDING_TYPE, BindingType.ATOMPUB.value());

      // Set the alfresco object factory
      parameter.put(SessionParameter.OBJECT_FACTORY_CLASS, "org.alfresco.cmis.client.impl.AlfrescoObjectFactoryImpl");

      // Create a session
      SessionFactory factory = SessionFactoryImpl.newInstance();
      Session session = factory.getRepositories(parameter).get(0).createSession();

      return session;
   }

   private static String translate(final String abstractPath, final Map<String, String> parameters) {
      String res = abstractPath;

      final Matcher matcher = PATTERN.matcher(abstractPath);

      while (matcher.find()) {
         final String var = matcher.group(0);
         final String metaKey = matcher.group(1);

         final String replacement = parameters.get(metaKey);
         if (replacement != null) {
            res = StringUtils.replace(res, var, replacement);
         } else {
            System.out.println("throw new PathTranslationException");
         }
      }

      return res;
   }

   private static CmisObject getCmisObject(Session session, String path) {
      CmisObject cmisObject = null;

      try {
         cmisObject = session.getObjectByPath(path);
      } catch (CmisObjectNotFoundException e) {
         // We are ignoring exception
      }

      return cmisObject;
   }

   /**
    * Build and validate the specified path.
    * 
    * @param cleanParentPath
    *            the cleanParentPath
    */
   private static void buildAndValidateFullPath(Session session, final String cleanParentPath) {
      String[] paths = StringUtils.split(cleanParentPath, "/");
      if (paths != null) {
         String path = "";
         CmisObject cmisObjectLastFounded = null;
         for (int i = 0; i < paths.length; i++) {
            path += "/" + paths[i];

            CmisObject cmisObject = getCmisObject(session, path);
            if (cmisObject == null) {
               Map<String, Object> properties = new HashMap<String, Object>();
               properties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
               properties.put(PropertyIds.NAME, paths[i]);
               Folder lastFoundedFolder = (Folder) cmisObjectLastFounded;
               lastFoundedFolder.createFolder(properties);
            } else {
               cmisObjectLastFounded = cmisObject;
            }
         }
      }
   }

   private static String cleanPath(String path) {
      return StringUtils.replaceEach(path, new String[] { "/app:company_home", "/cm:" }, new String[] { "", "/" });
   }

}
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Slow on update using WS

mrogers — Tue, 08 Jan 2013 13:14:53 GMT

See ALF-2713. There was a problem fixed in 3.3 timeframe and a note that the time still increases linearly.

Interestingly there were some timings associated with that fix, may be worth comparing your results with those.

Re: Slow on update using WS

marcosvasousa — Tue, 08 Jan 2013 19:49:11 GMT

Thanks.
Let me check if there is an patch/diff to apply to version 3.2 to resolve fast as possible.

We are currently installing Alfresco Community 3.4 to check this issue is resolved and to decide between:
> Migrate to Alfresco 4
> Upgrade to Alfresco 3.4
> Patch version 3.2 with fix from ALF-2713

While investigating Alfresco 4, I found that a document with 795 history:
getAlfrescoSession: 93 ms.
getCmisObject: 759 ms. - Using session.getObjectByPath
setContentStream: 147 ms.
updateProperties: 746 ms.
Total: 1746 ms.

Re: Slow on update using WS

marcosvasousa — Wed, 09 Jan 2013 08:27:07 GMT

We tested "> Upgrade to Alfresco 3.4" so we didn't found problems (no code upgrade/change is needed) and after 700 documents in history the time is below 200 ms.
So, we are now Upgranding to Alfresco 3.4.

topic Slow on update using WS in Alfresco Archive

Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS

Re: Slow on update using WS