Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Hyland Connect
- Enterprise Platforms
- Alfresco
- Alfresco Archive
- Re: Alfresco crashing during working hours
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Alfresco crashing during working hours
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-15-2016 07:01 PM
So, we are now experiencing this problem - Alfresco is very slow during working hours and finally crashes. Our setup is following:
Virtual server for applications (alfresco and share): 4 x 2.40 GHz, 9.5 GB RAM;
Database server: 4 x 2.40 GHz, 5GB RAM;
OS: Centos 6;
Alfresco version: Alfresco 4.2.e + Alvex Enterprise 2.1.2 plugin;
Users: <500;
Active users: 10-20;
<strong>Problem we are having</strong>
Alfresco documentation says, that with such server parameters we should be able to run server with 500 users or 50 concurrent.
During working hours, load average of server increases to ~30, CPU runs 100% and system becomes very slow. After running slow system crashed and starts throwing errors. I attach screenshot of server load:
[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/load_average.png[/img]
Those spikes in load average looks strange for me. When I am using the system when there is no load, I see 10-40% CPU utilization for 1 or 2 seconds after each click I make (home page, repository, my tasks, site page). If I press my completed tasks, there is 40% CPU utilization for about 2-3 seconds (I have only 135 tasks completed, other business users has a lot more tasks completed).
Some quotes from logs:
After Alfresco restart everything works fine until again load average reaches value of ~30.
<strong>Solutions we have tried</strong>
We have tried several solutions, which were not very successful:
1. Different servers for alfresco application and database (worked for couple months);
2. Tuning JVM, but no results;
Current JVM settings:
Do you have any other ideas which would be worth trying? Maybe you have any ideas how to identify which user actions are causing these spikes in load average?
Virtual server for applications (alfresco and share): 4 x 2.40 GHz, 9.5 GB RAM;
Database server: 4 x 2.40 GHz, 5GB RAM;
OS: Centos 6;
Alfresco version: Alfresco 4.2.e + Alvex Enterprise 2.1.2 plugin;
Users: <500;
Active users: 10-20;
<strong>Problem we are having</strong>
Alfresco documentation says, that with such server parameters we should be able to run server with 500 users or 50 concurrent.
During working hours, load average of server increases to ~30, CPU runs 100% and system becomes very slow. After running slow system crashed and starts throwing errors. I attach screenshot of server load:
[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/load_average.png[/img]
Those spikes in load average looks strange for me. When I am using the system when there is no load, I see 10-40% CPU utilization for 1 or 2 seconds after each click I make (home page, repository, my tasks, site page). If I press my completed tasks, there is 40% CPU utilization for about 2-3 seconds (I have only 135 tasks completed, other business users has a lot more tasks completed).
Some quotes from logs:
2016-02-11 09:17:16,038 INFO [webscripts.connector.RemoteClient] [http-apr-8080-exec-314] Error status 408 Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.springframework.extensions.webscripts.connector.RemoteClient.service(RemoteClient.java:1054)
at org.springframework.extensions.webscripts.connector.RemoteClient.service(RemoteClient.java:829)
at org.springframework.extensions.webscripts.connector.RemoteClient.call(RemoteClient.java:577)
at org.springframework.extensions.webscripts.connector.RemoteClient.call(RemoteClient.java:511)
at org.springframework.extensions.webscripts.connector.HttpConnector.call(HttpConnector.java:70)
at org.springframework.extensions.webscripts.RequestCachingConnector.call(RequestCachingConnector.java:90)
at org.springframework.extensions.webscripts.connector.AbstractConnector.call(AbstractConnector.java:116)
at org.springframework.extensions.webscripts.connector.AuthenticatingConnector.call(AuthenticatingConnector.java:92)
at org.alfresco.web.scripts.UserPreferences.getValue(UserPreferences.java:66)
at sun.reflect.GeneratedMethodAccessor1018.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
at org.mozilla.javascript.JavaMembers.get(JavaMembers.java:117)
at org.mozilla.javascript.NativeJavaObject.get(NativeJavaObject.java:113)
at org.mozilla.javascript.ScriptableObject.getProperty(ScriptableObject.java:1544)
at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1375)
at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1364)
at org.mozilla.javascript.gen.c5._c32(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js:1859)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:108)
at org.mozilla.javascript.gen.c5._c0(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js:1862)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2834)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.gen.c5.exec(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.springframework.extensions.webscripts.processor.JSScriptProcessor.executeScriptImpl(JSScriptProcessor.java:318)
at org.springframework.extensions.webscripts.processor.JSScriptProcessor.executeScript(JSScriptProcessor.java:192)
at org.springframework.extensions.webscripts.AbstractWebScript.executeScript(AbstractWebScript.java:1305)
at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:86)
at org.springframework.extensions.webscripts.PresentationContainer.executeScript(PresentationContainer.java:70)
at org.springframework.extensions.webscripts.LocalWebScriptRuntimeContainer.executeScript(LocalWebScriptRuntimeContainer.java:236)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:378)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:209)
at org.springframework.extensions.webscripts.WebScriptProcessor.executeBody(WebScriptProcessor.java:310)
at org.springframework.extensions.surf.render.AbstractProcessor.execute(AbstractProcessor.java:57)
at org.springframework.extensions.surf.render.RenderService.process(RenderService.java:599)
at org.springframework.extensions.surf.render.RenderService.renderSubComponent(RenderService.java:505)
at org.springframework.extensions.surf.render.RenderService.processComponent(RenderService.java:419)
at org.springframework.extensions.surf.render.RenderService.renderComponent(RenderService.java:942)
at org.springframework.extensions.surf.render.RenderService.renderRegionComponents(RenderService.java:900)
at org.springframework.extensions.surf.render.bean.RegionRenderer.body(RegionRenderer.java:89)
at org.springframework.extensions.surf.render.AbstractRenderer.render(AbstractRenderer.java:77)
at org.springframework.extensions.surf.render.RenderService.renderRegion(RenderService.java:851)
at org.springframework.extensions.directives.RegionDirectiveData.render(RegionDirectiveData.java:91)
at org.springframework.extensions.surf.extensibility.impl.ExtensibilityModelImpl.merge(ExtensibilityModelImpl.java:408)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.merge(AbstractExtensibilityDirective.java:169)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.execute(AbstractExtensibilityDirective.java:137)
at freemarker.core.Environment.visit(Environment.java:274)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:126)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment$3.render(Environment.java:246)
at org.springframework.extensions.surf.extensibility.impl.DefaultExtensibilityDirectiveData.render(DefaultExtensibilityDirectiveData.java:119)
at org.springframework.extensions.surf.extensibility.impl.ExtensibilityModelImpl.merge(ExtensibilityModelImpl.java:408)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.merge(AbstractExtensibilityDirective.java:169)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.execute(AbstractExtensibilityDirective.java:137)
at freemarker.core.Environment.visit(Environment.java:274)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:126)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.visit(Environment.java:406)
at freemarker.core.BodyInstruction.accept(BodyInstruction.java:93)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Macro$Context.runMacro(Macro.java:172)
at freemarker.core.Environment.visit(Environment.java:614)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:106)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.process(Environment.java:199)
at org.springframework.extensions.webscripts.processor.FTLTemplateProcessor.process(FTLTemplateProcessor.java:171)
at org.springframework.extensions.webscripts.WebTemplateProcessor.executeBody(WebTemplateProcessor.java:438)
at org.springframework.extensions.surf.render.AbstractProcessor.execute(AbstractProcessor.java:57)
at org.springframework.extensions.surf.render.RenderService.processTemplate(RenderService.java:721)
at org.springframework.extensions.surf.render.bean.TemplateInstanceRenderer.body(TemplateInstanceRenderer.java:140)
at org.springframework.extensions.surf.render.AbstractRenderer.render(AbstractRenderer.java:77)
at org.springframework.extensions.surf.render.bean.PageRenderer.body(PageRenderer.java:85)
at org.springframework.extensions.surf.render.AbstractRenderer.render(AbstractRenderer.java:77)
at org.springframework.extensions.surf.render.RenderService.renderPage(RenderService.java:762)
at org.springframework.extensions.surf.mvc.PageView.dispatchPage(PageView.java:411)
at org.springframework.extensions.surf.mvc.PageView.renderView(PageView.java:306)
at org.springframework.extensions.surf.mvc.AbstractWebFrameworkView.renderMergedOutputModel(AbstractWebFrameworkView.java:316)
at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:250)
at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1047)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:817)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:389)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.MTAuthenticationFilter.doFilter(MTAuthenticationFilter.java:74)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:168)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.CSRFFilter.doFilter(CSRFFilter.java:313)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.SSOAuthenticationFilter.doFilter(SSOAuthenticationFilter.java:378)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:1810)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
2016-02-11 09:22:34,322 ERROR [extensions.webscripts.AbstractRuntime] [http-apr-8080-exec-325] Exception from executeScript - redirecting to status template error: 01110055 Failed to execute script 'classpath*:alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js': 01110054 TypeError: Cannot read property "data" from null (jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js#37)
org.springframework.extensions.webscripts.WebScriptException: 01110055 Failed to execute script 'classpath*:alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js': 01110054 TypeError: Cannot read property "data" from null (jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js#37)
at org.springframework.extensions.webscripts.processor.JSScriptProcessor.executeScript(JSScriptProcessor.java:200)
at org.springframework.extensions.webscripts.AbstractWebScript.executeScript(AbstractWebScript.java:1305)
at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:86)
at org.springframework.extensions.webscripts.PresentationContainer.executeScript(PresentationContainer.java:70)
at org.springframework.extensions.webscripts.LocalWebScriptRuntimeContainer.executeScript(LocalWebScriptRuntimeContainer.java:236)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:378)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:209)
at org.springframework.extensions.webscripts.WebScriptProcessor.executeBody(WebScriptProcessor.java:310)
at org.springframework.extensions.surf.render.AbstractProcessor.execute(AbstractProcessor.java:57)
at org.springframework.extensions.surf.render.RenderService.process(RenderService.java:599)
at org.springframework.extensions.surf.render.RenderService.renderSubComponent(RenderService.java:505)
at org.springframework.extensions.surf.render.RenderService.processComponent(RenderService.java:419)
at org.springframework.extensions.surf.render.RenderService.renderComponent(RenderService.java:942)
at org.springframework.extensions.surf.render.RenderService.renderRegionComponents(RenderService.java:900)
at org.springframework.extensions.surf.render.bean.RegionRenderer.body(RegionRenderer.java:89)
at org.springframework.extensions.surf.render.AbstractRenderer.render(AbstractRenderer.java:77)
at org.springframework.extensions.surf.render.RenderService.renderRegion(RenderService.java:851)
at org.springframework.extensions.directives.RegionDirectiveData.render(RegionDirectiveData.java:91)
at org.springframework.extensions.surf.extensibility.impl.ExtensibilityModelImpl.merge(ExtensibilityModelImpl.java:408)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.merge(AbstractExtensibilityDirective.java:169)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.execute(AbstractExtensibilityDirective.java:137)
at freemarker.core.Environment.visit(Environment.java:274)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:126)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment$3.render(Environment.java:246)
at org.springframework.extensions.surf.extensibility.impl.DefaultExtensibilityDirectiveData.render(DefaultExtensibilityDirectiveData.java:119)
at org.springframework.extensions.surf.extensibility.impl.ExtensibilityModelImpl.merge(ExtensibilityModelImpl.java:408)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.merge(AbstractExtensibilityDirective.java:169)
at org.springframework.extensions.surf.extensibility.impl.AbstractExtensibilityDirective.execute(AbstractExtensibilityDirective.java:137)
at freemarker.core.Environment.visit(Environment.java:274)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:126)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.visit(Environment.java:406)
at freemarker.core.BodyInstruction.accept(BodyInstruction.java:93)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Macro$Context.runMacro(Macro.java:172)
at freemarker.core.Environment.visit(Environment.java:614)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:106)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.process(Environment.java:199)
at org.springframework.extensions.webscripts.processor.FTLTemplateProcessor.process(FTLTemplateProcessor.java:171)
at org.springframework.extensions.webscripts.WebTemplateProcessor.executeBody(WebTemplateProcessor.java:438)
at org.springframework.extensions.surf.render.AbstractProcessor.execute(AbstractProcessor.java:57)
at org.springframework.extensions.surf.render.RenderService.processTemplate(RenderService.java:721)
at org.springframework.extensions.surf.render.bean.TemplateInstanceRenderer.body(TemplateInstanceRenderer.java:140)
at org.springframework.extensions.surf.render.AbstractRenderer.render(AbstractRenderer.java:77)
at org.springframework.extensions.surf.render.bean.PageRenderer.body(PageRenderer.java:85)
at org.springframework.extensions.surf.render.AbstractRenderer.render(AbstractRenderer.java:77)
at org.springframework.extensions.surf.render.RenderService.renderPage(RenderService.java:762)
at org.springframework.extensions.surf.mvc.PageView.dispatchPage(PageView.java:411)
at org.springframework.extensions.surf.mvc.PageView.renderView(PageView.java:306)
at org.springframework.extensions.surf.mvc.AbstractWebFrameworkView.renderMergedOutputModel(AbstractWebFrameworkView.java:316)
at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:250)
at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1047)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:817)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:412)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
at org.tuckey.web.filters.urlrewrite.NormalRewrittenUrl.doRewrite(NormalRewrittenUrl.java:213)
at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:171)
at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:389)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.MTAuthenticationFilter.doFilter(MTAuthenticationFilter.java:74)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:168)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.CSRFFilter.doFilter(CSRFFilter.java:313)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.site.servlet.SSOAuthenticationFilter.doFilter(SSOAuthenticationFilter.java:378)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:1810)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.springframework.extensions.webscripts.WebScriptException: 01110054 TypeError: Cannot read property "data" from null (jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js#37)
at org.springframework.extensions.webscripts.processor.JSScriptProcessor.executeScriptImpl(JSScriptProcessor.java:328)
at org.springframework.extensions.webscripts.processor.JSScriptProcessor.executeScript(JSScriptProcessor.java:192)
… 106 more
Caused by: org.mozilla.javascript.EcmaError: TypeError: Cannot read property "data" from null (jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js#37)
at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3350)
at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3340)
at org.mozilla.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3356)
at org.mozilla.javascript.ScriptRuntime.typeError2(ScriptRuntime.java:3375)
at org.mozilla.javascript.ScriptRuntime.undefReadError(ScriptRuntime.java:3388)
at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1362)
at org.mozilla.javascript.gen.c5._c2(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js:37)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:108)
at org.mozilla.javascript.gen.c5._c3(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js:41)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:108)
at org.mozilla.javascript.gen.c5._c33(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js:1871)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:108)
at org.mozilla.javascript.gen.c5._c0(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js:2093)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2834)
at org.mozilla.javascript.gen.c5.call(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.mozilla.javascript.gen.c5.exec(jar:file:/opt/alfresco/tomcat/webapps/share/WEB-INF/lib/alvex-share-4.2.d-generic-ui-ee-share-1.16.jar!/alfresco/web-extension/site-webscripts/org/alfresco/share/header/share-header.get.js)
at org.springframework.extensions.webscripts.processor.JSScriptProcessor.executeScriptImpl(JSScriptProcessor.java:318)
… 107 more
After Alfresco restart everything works fine until again load average reaches value of ~30.
<strong>Solutions we have tried</strong>
We have tried several solutions, which were not very successful:
1. Different servers for alfresco application and database (worked for couple months);
2. Tuning JVM, but no results;
Current JVM settings:
JAVA_OPTS="-XX:MaxPermSize=256m -Xss1024K -Xms1G -Xmx8G -XX:+DisableExplicitGC -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Dalfresco.home=/opt/alfresco -Dcom.sun.management.jmxremote -Dsun.security.ssl.allowUnsafeRenegotiation=true -server"Do you have any other ideas which would be worth trying? Maybe you have any ideas how to identify which user actions are causing these spikes in load average?
Labels:
- Labels:
-
Archive
12 REPLIES 12
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2016 01:03 AM
You may want to take a look at the very good blogpost - http://blyx.com/2015/02/05/alfresco-tuning-shortlist/ which may give you some direction to proceed.
Hope this helps.
Hope this helps.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2016 06:03 AM
Thank you for the link. We have already tried most of the stuff mentioned in the link, but there is some new things at the end of the article. Hope those will help.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2016 04:23 AM
And if you are using alfresco enterprise, then I suggest you call alfresco support.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2016 06:03 AM
We are using Alfresco Community version.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-17-2016 04:40 PM
If you are familiar with htop run that and it will give you a break down of what loads you are running on your CPU's and what process is running.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-23-2016 07:19 AM
We have installed htop and checked the data. Looks like there are ~180 threads running. Most of them (~90%) are /opt/alfresco/java/bin/java. They are using ~0 or 1 % of CPU. Every couple seconds some (1, 2, 3, 4 or more) of /opt/alfresco/java/bin/java threads starts using (10% - 60%) of CPU core. So if there is more threads, eventually it reaches 400% CPU and then we are having too high load average.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-23-2016 07:27 AM
I also attach htop data exactly at the moment of admin user loading the homepage. Before and after, CPU load was ~5%.
[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/htop.png[/img]
[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/htop.png[/img]
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-23-2016 08:41 AM
One more update. I was checking load with htop and noticed, that some threads takes high CPU for long time. For example in the screenshot there is 5 running threads. Each of them were running for at least 1 minute at about 80% CPU utilization. Some of them (for example 22240, 22282) were running for longer - 5 minutes and more and is now still running while I am writing this post.
Is there a way to dig deeper and understand why those threads are taking so long and what functions are users using? Myself I can not trigger a thread while using system which would have this CPU utilization and would take so long.
[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/htop2.png[/img]
Is there a way to dig deeper and understand why those threads are taking so long and what functions are users using? Myself I can not trigger a thread while using system which would have this CPU utilization and would take so long.
[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/htop2.png[/img]
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-23-2016 12:15 PM
Could you give the steps of tuning your JVM?
You should be modifying these files.
/alfrescoinstall/tomcat/bin/setenv.sh
/alfrescoinstall/tomcat/scripts/ctl.sh
You should be modifying these files.
/alfrescoinstall/tomcat/bin/setenv.sh
/alfrescoinstall/tomcat/scripts/ctl.sh
Related Content
- Customize the Alfresco system source code in Alfresco Forum
- Alfresco BPMN - Typeahead Field - How to work with this in Alfresco Forum
- A namespace prefix is not registered - Simple OCR in Alfresco Forum
- How to enable Ticket base authentication ? in Alfresco Forum
- DOCKER TRANSFORM with libreoffice - issue in Alfresco Forum
Getting started
Tags
Find what you came for
We want to make your experience in Hyland Connect as valuable as possible, so we put together some helpful links.
