cancel
Showing results for 
Search instead for 
Did you mean: 

Exceptionally slow 5.0.c performace

globaladvisors_
Champ in-the-making
Champ in-the-making
Hi all

We recently upgraded to Alfresco 4.0.c from Alfresco 4.2.e

Alfresco is running on a dedicated server running one XEN VM:

Alfresco 5.0.c
XEN 4.4 VM
14GB RAM 6 pinned CPUs
JAVA_OPTS="-server -Xss1024K -Xms4G -Xmx6G -XX:MaxPermSize=256M -XX:NewSize=512M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=80 -Dalfresco.home=/opt/alfresco -Dcom.sun.management.jmxremote -Dsun.security.ssl.allowUnsafeRenegotiation=true"
Debian Testing

We have struggled with performance issues since the upgrade.

Pstree output:

<blockquote>
init-+-bacula-fd—3*[{bacula-fd}]

     |-cgmanager

     |-clamd—{clamd}

     |-cron

     |-dbus-daemon

     |-freshclam

     |-getty

     |-java-+-oosplash-+-.soffice.bin-+-{.soffice.bin}

     |      |          |              |-{OfficeIPCThread}

     |      |          |              |-{binaryurpReader}

     |      |          |              |-{binaryurpWriter}

     |      |          |              `-{rtl_cache_wsupd}

     |      |          |-{oosplash}

     |      |          `-{rtl_cache_wsupd}

     |      `-183*[{java}]

     |-miniserv.pl

     |-mysqld_safe—mysqld—27*[{mysqld}]

     |-postgres—16*[postgres]

     |-rsyslogd-+-{in:imklog}

     |          |-{in:imuxsock}

     |          `-{rs:main Q:Reg}

     |-sshd-+-sshd—bash—pstree

     |      `-sshd—bash

     |-systemd-logind

     `-udevd

</blockquote>

Java processes are typically numbering in the hundreds and postgres often hits high numbers.

iotop often shows many instances of:

java -Djava.util.logging.config.file=/op~.apache.catalina.startup.Bootstrap start


Errors noticed in the alfresco.log:

This is a frequently recurring error
<blockquote>
09:58:08,899 ERROR [org.quartz.core.ErrorLogger] [DefaultScheduler_Worker-8] Job (DEFAULT.taggingStartupJobDetail threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.RuntimeException: Unable to update the tag scopes.]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:563)
Caused by: java.lang.RuntimeException: Unable to update the tag scopes.
        at org.alfresco.repo.tagging.UpdateTagScopesActionExecuter.executeImpl(UpdateTagScopesActionExecuter.java:288)
        at org.alfresco.repo.action.executer.ActionExecuterAbstractBase.execute(ActionExecuterAbstractBase.java:265)
        at org.alfresco.repo.action.ActionServiceImpl.directActionExecution(ActionServiceImpl.java:839)
        at org.alfresco.repo.action.ActionServiceImpl.executeActionImpl(ActionServiceImpl.java:739)
        at org.alfresco.repo.action.ActionServiceImpl.executeAction(ActionServiceImpl.java:573)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at org.alfresco.repo.security.permissions.impl.AlwaysProceedMethodInterceptor.invoke(AlwaysProceedMethodInterceptor.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:46)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:159)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at com.sun.proxy.$Proxy37.executeAction(Unknown Source)
        at org.alfresco.repo.tagging.UpdateTagScopesQuartzJob.execute(UpdateTagScopesQuartzJob.java:145)
        at org.alfresco.repo.tagging.UpdateTagScopesQuartzJob.execute(UpdateTagScopesQuartzJob.java:92)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
        … 1 more
Caused by: org.alfresco.error.AlfrescoRuntimeException: 002911510 Node type is not recognised: {http://www.alfresco.org/model/website/1.0}websiteContainer
        at org.alfresco.repo.node.integrity.IncompleteNodeTagger.processNode(IncompleteNodeTagger.java:422)
        at org.alfresco.repo.node.integrity.IncompleteNodeTagger.access$100(IncompleteNodeTagger.java:60)
        at org.alfresco.repo.node.integrity.IncompleteNodeTagger$1.doWork(IncompleteNodeTagger.java:399)
        at org.alfresco.repo.node.integrity.IncompleteNodeTagger$1.doWork(IncompleteNodeTagger.java:391)
        at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:548)
        at org.alfresco.repo.node.integrity.IncompleteNodeTagger.beforeCommit(IncompleteNodeTagger.java:405)
        at org.alfresco.util.transaction.TransactionSupportUtil$TransactionSynchronizationImpl.doBeforeCommit(TransactionSupportUtil.java:535)
        at org.alfresco.util.transaction.TransactionSupportUtil$TransactionSynchronizationImpl.doBeforeCommit(TransactionSupportUtil.java:514)
        at org.alfresco.util.transaction.TransactionSupportUtil$TransactionSynchronizationImpl.beforeCommit(TransactionSupportUtil.java:479)
        at org.springframework.transaction.support.TransactionSynchronizationUtils.triggerBeforeCommit(TransactionSynchronizationUtils.java:95)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerBeforeCommit(AbstractPlatformTransactionManager.java:925)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:738)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:475)
        at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:482)
        at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:476)
        at org.alfresco.repo.tagging.UpdateTagScopesActionExecuter$1.doWork(UpdateTagScopesActionExecuter.java:222)
        at org.alfresco.repo.tagging.UpdateTagScopesActionExecuter$1.doWork(UpdateTagScopesActionExecuter.java:216)
        at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:548)
        at org.alfresco.repo.tagging.UpdateTagScopesActionExecuter.executeImpl(UpdateTagScopesActionExecuter.java:215)
        … 27 more
</blockquote>

From our share.log we get a number of read timeouts:

<blockquote>
10:56:57,402 INFO  [org.springframework.extensions.webscripts.connector.RemoteClient] [http-apr-8080-exec-64] Exception calling (GET) http://localhost:8080/alfresco/s/api/node/workspace/SpacesStore/d33709b6-455c-4bff-b1ac-dcce35b75206...
10:56:57,402 INFO  [org.springframework.extensions.webscripts.connector.RemoteClient] [http-apr-8080-exec-64] Error status 408 Read timed out
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(Unknown Source)
        at java.net.SocketInputStream.read(Unknown Source)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
        at sun.reflect.GeneratedMethodAccessor479.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
        at com.sun.proxy.$Proxy280.receiveResponseHeader(Unknown Source)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
        at org.springframework.extensions.webscripts.connector.RemoteClient.service(RemoteClient.java:1095)
        at org.springframework.extensions.webscripts.connector.RemoteClient.call(RemoteClient.java:745)
        at org.springframework.extensions.webscripts.connector.HttpConnector.call(HttpConnector.java:143)
        at org.springframework.extensions.webscripts.connector.AuthenticatingConnector.call(AuthenticatingConnector.java:293)
        at org.springframework.extensions.webscripts.servlet.mvc.EndPointProxyController.handleRequestInternal(EndPointProxyController.java:326)
        at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
        at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:919)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:844)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:748)
        at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:486)
        at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:411)
        at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:338)
        at org.tuckey.web.filters.urlrewrite.NormalRewrittenUrl.doRewrite(NormalRewrittenUrl.java:213)
        at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:171)
        at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
        at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
        at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:389)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.alfresco.web.site.servlet.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:168)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.alfresco.web.site.servlet.CSRFFilter.doFilter(CSRFFilter.java:322)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.alfresco.web.site.servlet.SSOAuthenticationFilter.doFilter(SSOAuthenticationFilter.java:394)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2441)
        at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2430)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
</blockquote>
1 REPLY 1

mlagneaux
Champ on-the-rise
Champ on-the-rise
You can take a look at this presentation from Romain Guinot at Alfresco Summit 2013 in Barcelona.
I think it should help you.

https://summit.alfresco.com/barcelona/sessions/performance-troubleshooting-tooling
Getting started

Tags


Find what you came for

We want to make your experience in Hyland Connect as valuable as possible, so we put together some helpful links.