Slow on update using WS

[marcosvasousa]

I found that the line is very slow final UpdateResult[] updateResults = repositoryServiceSoapBindingStub.update(cml);
It takes at first call 13 seconds and subsequentes call takes 5 seconds.

I put MYSQL logging query and no slow query founded. Also, I indexed the repository. No solution give to me good result.
Is it normal to take always between 5 - 13 seconds while update document? How can I speed my code?

Notes: The document created contains Version Label: 1589. If first tried search forum and no solution founded. Maybe consider this https://forums.alfresco.com/en/viewtopic.php?f=53&t=2056 as same!

Found below the code used. Please help me.

import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.rmi.RemoteException;
import java.text.MessageFormat;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.alfresco.webservice.repository.RepositoryFault;
import org.alfresco.webservice.repository.RepositoryServiceSoapBindingStub;
import org.alfresco.webservice.repository.UpdateResult;
import org.alfresco.webservice.types.CML;
import org.alfresco.webservice.types.CMLCreate;
import org.alfresco.webservice.types.CMLUpdate;
import org.alfresco.webservice.types.CMLWriteContent;
import org.alfresco.webservice.types.ContentFormat;
import org.alfresco.webservice.types.NamedValue;
import org.alfresco.webservice.types.Node;
import org.alfresco.webservice.types.ParentReference;
import org.alfresco.webservice.types.Predicate;
import org.alfresco.webservice.types.Reference;
import org.alfresco.webservice.types.Store;
import org.alfresco.webservice.util.AuthenticationUtils;
import org.alfresco.webservice.util.Constants;
import org.alfresco.webservice.util.ISO9075;
import org.alfresco.webservice.util.Utils;
import org.alfresco.webservice.util.WebServiceFactory;
import org.apache.commons.lang.StringUtils;

public class UpdateAlfresco {

   private static final Pattern PATTERN = Pattern.compile("\\{([A-Za-z_]+)\\}");

   private static final String name = "http://www.alfresco.org/model/content/1.0}name";
   private static final String version = "http://www.alfresco.org/model/content/1.0}versionLabel";
   private static final String modified = "http://www.alfresco.org/model/content/1.0}modified";
   private static final String ticketServiceAddress = "http://MACHINE:8280/alfresco/service/api/login/ticket";
   private static final String endPointAddress = "http://MACHINE:8280/alfresco/api";
   private static final String user = "USER";
   private static final String password = "PWD";
   private static final String storeScheme = "workspace";
   private static final String storeAddress = "spacesStore";
   private static final String rootPath = "/app:company_home/cm:IFE";
   private static final String modelTypeName = "imp:base";
   private static final String modelNameSpace = "http://ola.com/model/content/1.0";
   private static final Store store = new Store(storeScheme, storeAddress);

   /**
    * @param args
    */
   public static void main(String[] args) throws Exception {
      WebServiceFactory.setEndpointAddress(endPointAddress);
      AuthenticationUtils.startSession(user, password);
      String ticket = AuthenticationUtils.getTicket();

      Map<String, String> filteredMetadataProps = new HashMap<String, String>();
      filteredMetadataProps.put("tipo_documento", "my_document");
      filteredMetadataProps.put("entity", "182178");

      InputStream dataStream = new FileInputStream(new File("C:\\alfresco\\MyDoc.pdf"));

      createDocument("/cm:Entities/cm:{entity}/cm:EIE", "my_document", "application/pdf", filteredMetadataProps, dataStream);
   }

   public static void createDocument(String path, String type, String contentType, Map<String, String> metadata, InputStream dataStream) throws Exception {
      // CML node identifier.
      String id = "bbff85d8-9980-4a30-9ab9-0f80e7cc0cad";

      // Translate path.
      String beforeEncodePath = rootPath + translate(path, new HashMap<String, String>(metadata));
      String concretePath = DocumentBrokerUtils.encodePath(beforeEncodePath);

      final String documentName = DocumentBrokerUtils.getNodeName(concretePath);

      // Create properties.
      final List<NamedValue> contentProperties = DocumentBrokerUtils.createProperties(metadata, modelNameSpace);
      contentProperties.add(Utils.createNamedValue(Constants.PROP_NAME, ISO9075.decode(documentName)));

      final CML cml = new CML();
      final CMLWriteContent writeContent = new CMLWriteContent();

      // Verify if document already exists.
      Node node = getNodeFromRepository(new Reference(store, null, concretePath));

      System.out.println(MessageFormat.format("Overriding existing document: {0}", concretePath));

      Reference reference = new Reference(store, id, concretePath);
      Node nodeId = getNodeFromRepository(reference);
      if (nodeId != null) {
         System.out.println("Using NodeId");

         final Predicate predicate = new Predicate(new Reference[] { reference }, store, null);

         final CMLUpdate update = new CMLUpdate(contentProperties.toArray(new NamedValue[0]), predicate, id);
         cml.setUpdate(new CMLUpdate[] { update });
         writeContent.setWhere(predicate);
      } else {
         final Predicate predicate = new Predicate(new Reference[] { node.getReference() }, store, null);
         final CMLUpdate update = new CMLUpdate(contentProperties.toArray(new NamedValue[0]), predicate, null);
         cml.setUpdate(new CMLUpdate[] { update });
         writeContent.setWhere(predicate);
      }

      // Write the document content.
      writeContent.setProperty(Constants.PROP_CONTENT);
      writeContent.setFormat(new ContentFormat(contentType, "UTF-8"));
      writeContent.setContent(DocumentBrokerUtils.readFile(dataStream));
      cml.setWriteContent(new CMLWriteContent[] { writeContent });

      RepositoryServiceSoapBindingStub repositoryServiceSoapBindingStub = WebServiceFactory.getRepositoryService();
      long ini = System.currentTimeMillis();
      final UpdateResult[] updateResults = repositoryServiceSoapBindingStub.update(cml);
      System.out.println("update in " + (System.currentTimeMillis() - ini) + " ms. UUID: " + updateResults[0].getDestination().getUuid());
   }

   private static String translate(final String abstractPath, final Map<String, String> parameters) {

      String res = abstractPath;

      final Matcher matcher = PATTERN.matcher(abstractPath);

      while (matcher.find()) {
         final String var = matcher.group(0);
         final String metaKey = matcher.group(1);

         final String replacement = parameters.get(metaKey);
         if (replacement != null) {
            res = StringUtils.replace(res, var, replacement);
         } else {
            System.out.println("throw new PathTranslationException");
         }
      }

      System.out.println("translate " + res);

      return res;
   }
   
   private static void buildAndValidateFullPath(final String path) throws Exception {
      final String parentPath = DocumentBrokerUtils.getParentPath(path);

      if (parentPath == null) {
         System.out.println("throw new InvalidPathException()");
      } else {
         try {
            // Get the current path.
            WebServiceFactory.getRepositoryService().get(new Predicate(new Reference[] { new Reference(store, null, parentPath) }, store, null));
         } catch (final Exception e) {
            buildAndValidateFullPath(parentPath);
            createFolder(DocumentBrokerUtils.getParentPath(parentPath), DocumentBrokerUtils.getNodeName(parentPath));
         }
      }
   }

   private static Reference createFolder(final String path, final String folderName) throws Exception {

      // Get Parent Reference.
      final ParentReference parentReference = new ParentReference(store, null, path, Constants.ASSOC_CONTAINS, Constants.createQNameString(Constants.NAMESPACE_CONTENT_MODEL,
            ISO9075.decode(folderName)));

      final NamedValue[] contentProps = new NamedValue[1];
      contentProps[0] = Utils.createNamedValue(Constants.PROP_NAME, ISO9075.decode(folderName));
      final CMLCreate create = new CMLCreate("1", parentReference, null, null, null, Constants.TYPE_FOLDER, contentProps);

      final CML cml = new CML();
      cml.setCreate(new CMLCreate[] { create });

      final UpdateResult[] updateResult = WebServiceFactory.getRepositoryService().update(cml);
      return updateResult[0].getDestination();
   }

   private static Node getNodeFromRepository(final Reference reference) {
      Node node = null;
      try {
         final Node[] nodes = WebServiceFactory.getRepositoryService().get(new Predicate(new Reference[] { reference }, store, null));
         if (nodes.length > 0) {
            node = nodes[0];
         }
      } catch (final Exception e) {
         // do nothing, document does not exists.
      }
      return node;
   }

   private static Map<String, String> getProperties(final Store store, final Reference reference) throws RepositoryFault, RemoteException {
      final Map<String, String> properties = new HashMap<String, String>();

      final Node[] nodes = WebServiceFactory.getRepositoryService().get(new Predicate(new Reference[] { reference }, store, null));

      for (final NamedValue namedValue : nodes[0].getProperties()) {
         properties.put(namedValue.getName(), namedValue.getValue());
      }

      return properties;
   }

}
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[marcosvasousa]

Please, can someone give to me any clue?

[marcosvasousa]

I tried alfresco version 4 enterprise trial and founded that updating an document with history/versions tooks to long also!

See numbers:
Alfresco 4 Enterprise Trial:
659 ms while updating an document with 39 history
1583 ms while updating an document with 185 history

Alfresco 3 Community:
722 ms while updating an document with 39 history
5289 ms while updating an document with 185 history

Is this normal numbers? Alfresco looks to be slow while more documents are being created!

[mrogers]

Well at least alfresco 4 is faster.

Yes there seems to be an issue there.    Not sure why document history is having that effect.   However your system still seems to
be running slowly.   First thing to check is the amount of RAM configured.

Also I suggest you don't continue to use a deprecated API.

[marcosvasousa]

Yes, looks like Alfresco 4 is very fast!

I read the http://chemistry.apache.org/java/0.4.0/maven/apidocs/org/apache/chemistry/opencmis/client/api/Docume... and found "boolean refresh". If use false Alfresco 4 is fast:

Alfresco 4 Enterprise Trial (Linux 64 bits):
329 ms while updating an document with 39 history
635 ms while updating an document with 185 history
…
752 ms while updating an document with 250 history
…
With 1000 documents without refresh, the times decrease from 7 seconds to 3.8 seconds.

We are under JVM memory: "server -Xss1M -Xms3G -Xmx6G -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:NewSize=1G -XX:MaxPermSize=256M -XX:CMSInitiatingOccupancyFraction=80 -"
so I don't think RAM is problem!!!
The machine has 8GB RAM.
Our Linux machine monitoring didn't indicates problems related to memory or disk!

Let me post my new code, so I really believe no longer using deprecated API in Alfresco 4!

[marcosvasousa]

Below found complete code used.
All my test showed to me that Alfresco will be slow while updating an document with version. Strange why document history do this effect!
What do you advice to me? Remember we are using Alfresco 3 Community in production.
Any additional information that can I provide to you?

package com.xpto.alfresco4;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.text.MessageFormat;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.chemistry.opencmis.client.api.CmisObject;
import org.apache.chemistry.opencmis.client.api.Document;
import org.apache.chemistry.opencmis.client.api.Folder;
import org.apache.chemistry.opencmis.client.api.Session;
import org.apache.chemistry.opencmis.client.api.SessionFactory;
import org.apache.chemistry.opencmis.client.runtime.SessionFactoryImpl;
import org.apache.chemistry.opencmis.commons.PropertyIds;
import org.apache.chemistry.opencmis.commons.SessionParameter;
import org.apache.chemistry.opencmis.commons.data.ContentStream;
import org.apache.chemistry.opencmis.commons.enums.BindingType;
import org.apache.chemistry.opencmis.commons.enums.VersioningState;
import org.apache.chemistry.opencmis.commons.exceptions.CmisObjectNotFoundException;
import org.apache.commons.lang.StringUtils;

public class CreateDocumentUsingCMIS {

   private static final Pattern PATTERN = Pattern.compile("\\{([A-Za-z_]+)\\}");

   private static final String rootPath = "/app:company_home/cm:XptoFE";

   /**
    * @param args
    */
   public static void main(String[] args) throws Exception {
      for (int i = 0; i < 1000; i++) {
         System.out.println(i);
         elementosInformativo();
      }
   }

   private static void elementosInformativo() throws Exception {
      Map<String, String> filteredMetadataProps = new HashMap<String, String>();
      filteredMetadataProps.put("tipo_documento", "elementos_informativos_entidade");
      filteredMetadataProps.put("entidade", "182179");// 182178

      InputStream dataStream = new FileInputStream(new File("C:\\Doc.pdf"));

      long ini = System.currentTimeMillis();
      createDocument("/cm:Entidades/cm:{entidade}/cm:ElementosInformativosEntidade", "elementos_informativos_entidade", "application/pdf", filteredMetadataProps, true, dataStream, true);
      System.out.println("Tempo Elementos Informativo: " + (System.currentTimeMillis() - ini) + " milisegundos.");
   }

   public static void createDocument(final String path, final String type, final String contentType, final Map<String, String> metadata, final boolean supportsVersions,
         final InputStream dataStream, final boolean override) throws Exception {
      // Translate path.
      final String beforeEncodePath = rootPath + translate(path, new HashMap<String, String>(metadata));
      final String documentName = DocumentBrokerUtils.getNodeName(beforeEncodePath);
      final String parentPath = DocumentBrokerUtils.getParentPath(beforeEncodePath);
      final String cleanParentPath = cleanPath(parentPath);

      Session session = getAlfrescoSession();
      CmisObject cmisObject = getCmisObject(session, cleanPath(beforeEncodePath));

      Map<String, Object> properties = new HashMap<String, Object>();
      properties.put(PropertyIds.OBJECT_TYPE_ID, "D:imp:documento");
      properties.put(PropertyIds.NAME, documentName);
      for (final Entry<String, String> entry : metadata.entrySet()) {
         properties.put("imp:" + entry.getKey(), entry.getValue());
      }

      System.out.println(properties);

      byte[] bytes = DocumentBrokerUtils.readFile(dataStream);
      ContentStream contentStream = session.getObjectFactory().createContentStream(documentName, (long) (bytes.length), contentType, new ByteArrayInputStream(bytes));

      if (cmisObject == null) {
         // Criar o ficheiro com versionamento
         System.out.println(MessageFormat.format("Creating new document: {0}", beforeEncodePath));

         // Build and validate path.
         buildAndValidateFullPath(session, cleanParentPath);

         CmisObject cmisObjectFolder = session.getObjectByPath(cleanParentPath);
         Folder folder = (Folder) cmisObjectFolder;
         folder.createDocument(properties, contentStream, VersioningState.MAJOR);
      } else {
         // Actualizar o ficheiro com versionamento
         System.out.println(MessageFormat.format("Overriding existing document: {0}", beforeEncodePath));

         Document document = (Document) cmisObject;
         document.setContentStream(contentStream, true, false);
         cmisObject.updateProperties(properties, false);
      }
   }

   private static Session getAlfrescoSession() {
      // 1. Authentication Session
      Map<String, String> parameter = new HashMap<String, String>();

      // Set the user credentials
      parameter.put(SessionParameter.USER, "admin");
      parameter.put(SessionParameter.PASSWORD, "PASSWORD");

      // Specify the connection settings
      parameter.put(SessionParameter.ATOMPUB_URL, "http://HOST:PORT/alfresco/service/cmis");
      parameter.put(SessionParameter.BINDING_TYPE, BindingType.ATOMPUB.value());

      // Set the alfresco object factory
      parameter.put(SessionParameter.OBJECT_FACTORY_CLASS, "org.alfresco.cmis.client.impl.AlfrescoObjectFactoryImpl");

      // Create a session
      SessionFactory factory = SessionFactoryImpl.newInstance();
      Session session = factory.getRepositories(parameter).get(0).createSession();

      return session;
   }

   private static String translate(final String abstractPath, final Map<String, String> parameters) {
      String res = abstractPath;

      final Matcher matcher = PATTERN.matcher(abstractPath);

      while (matcher.find()) {
         final String var = matcher.group(0);
         final String metaKey = matcher.group(1);

         final String replacement = parameters.get(metaKey);
         if (replacement != null) {
            res = StringUtils.replace(res, var, replacement);
         } else {
            System.out.println("throw new PathTranslationException");
         }
      }

      return res;
   }

   private static CmisObject getCmisObject(Session session, String path) {
      CmisObject cmisObject = null;

      try {
         cmisObject = session.getObjectByPath(path);
      } catch (CmisObjectNotFoundException e) {
         // We are ignoring exception
      }

      return cmisObject;
   }

   /**
    * Build and validate the specified path.
    * 
    * @param cleanParentPath
    *            the cleanParentPath
    */
   private static void buildAndValidateFullPath(Session session, final String cleanParentPath) {
      String[] paths = StringUtils.split(cleanParentPath, "/");
      if (paths != null) {
         String path = "";
         CmisObject cmisObjectLastFounded = null;
         for (int i = 0; i < paths.length; i++) {
            path += "/" + paths[i];

            CmisObject cmisObject = getCmisObject(session, path);
            if (cmisObject == null) {
               Map<String, Object> properties = new HashMap<String, Object>();
               properties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
               properties.put(PropertyIds.NAME, paths[i]);
               Folder lastFoundedFolder = (Folder) cmisObjectLastFounded;
               lastFoundedFolder.createFolder(properties);
            } else {
               cmisObjectLastFounded = cmisObject;
            }
         }
      }
   }

   private static String cleanPath(String path) {
      return StringUtils.replaceEach(path, new String[] { "/app:company_home", "/cm:" }, new String[] { "", "/" });
   }

}
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[mrogers]

See ALF-2713.    There was a problem fixed in 3.3 timeframe and a note that the time still increases linearly.   

Interestingly there were some timings associated with that fix,  may be worth comparing your results with those.

[marcosvasousa]

Thanks.
Let me check if there is an patch/diff to apply to version 3.2 to resolve fast as possible.

We are currently installing Alfresco Community 3.4 to check this issue is resolved and to decide between:
> Migrate to Alfresco 4
> Upgrade to Alfresco 3.4
> Patch version 3.2 with fix from ALF-2713

While investigating Alfresco 4, I found that a document with 795 history:
getAlfrescoSession: 93 ms.
getCmisObject: 759 ms. - Using session.getObjectByPath
setContentStream: 147 ms.
updateProperties: 746 ms.
Total: 1746 ms.

[marcosvasousa]

We tested "> Upgrade to Alfresco 3.4" so we didn't found problems (no code upgrade/change is needed) and after 700 documents in history the time is below 200 ms.
So, we are now Upgranding to Alfresco 3.4.