topic Re: Alfresco crashing during working hours in Alfresco Archive

Alfresco crashing during working hours

hostman — Tue, 16 Feb 2016 00:01:37 GMT

So, we are now experiencing this problem - Alfresco is very slow during working hours and finally crashes. Our setup is following:Virtual server for applications (alfresco and share): 4 x 2.40 GHz, 9.5 GB RAM;Database server: 4 x 2.40 GHz, 5GB RAM;OS: Centos 6;Alfresco version: Alfresco 4.2.e + Alve

Re: Alfresco crashing during working hours

romschn — Tue, 16 Feb 2016 06:03:19 GMT

You may want to take a look at the very good blogpost - http://blyx.com/2015/02/05/alfresco-tuning-shortlist/ which may give you some direction to proceed.

Hope this helps.

Re: Alfresco crashing during working hours

mrogers — Tue, 16 Feb 2016 09:23:42 GMT

And if you are using alfresco enterprise, then I suggest you call alfresco support.

Re: Alfresco crashing during working hours

hostman — Tue, 16 Feb 2016 11:03:18 GMT

Thank you for the link. We have already tried most of the stuff mentioned in the link, but there is some new things at the end of the article. Hope those will help.

Re: Alfresco crashing during working hours

hostman — Tue, 16 Feb 2016 11:03:58 GMT

We are using Alfresco Community version.

Re: Alfresco crashing during working hours

eswbitto — Wed, 17 Feb 2016 21:40:01 GMT

If you are familiar with htop run that and it will give you a break down of what loads you are running on your CPU's and what process is running.

Re: Alfresco crashing during working hours

hostman — Tue, 23 Feb 2016 12:19:20 GMT

We have installed htop and checked the data. Looks like there are ~180 threads running. Most of them (~90%) are /opt/alfresco/java/bin/java. They are using ~0 or 1 % of CPU. Every couple seconds some (1, 2, 3, 4 or more) of /opt/alfresco/java/bin/java threads starts using (10% - 60%) of CPU core. So if there is more threads, eventually it reaches 400% CPU and then we are having too high load average.

Re: Alfresco crashing during working hours

hostman — Tue, 23 Feb 2016 12:27:00 GMT

I also attach htop data exactly at the moment of admin user loading the homepage. Before and after, CPU load was ~5%.

[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/htop.png[/img]

Re: Alfresco crashing during working hours

hostman — Tue, 23 Feb 2016 13:41:28 GMT

One more update. I was checking load with htop and noticed, that some threads takes high CPU for long time. For example in the screenshot there is 5 running threads. Each of them were running for at least 1 minute at about 80% CPU utilization. Some of them (for example 22240, 22282) were running for longer - 5 minutes and more and is now still running while I am writing this post.
Is there a way to dig deeper and understand why those threads are taking so long and what functions are users using? Myself I can not trigger a thread while using system which would have this CPU utilization and would take so long.

[img=640x480]https://cdn-forums.alfresco.com/sites/forums/files/htop2.png[/img]

Re: Alfresco crashing during working hours

eswbitto — Tue, 23 Feb 2016 17:15:00 GMT

Could you give the steps of tuning your JVM?

You should be modifying these files.

/alfrescoinstall/tomcat/bin/setenv.sh
/alfrescoinstall/tomcat/scripts/ctl.sh

Re: Alfresco crashing during working hours

hostman — Wed, 24 Feb 2016 08:21:49 GMT

I have been playing with parameters in ctl.sh for ages. None of the customizations gives good results. My /alfrescoinstall/tomcat/scripts/ctl.sh at the moment has:


…
export JAVA_OPTS="-XX:MaxPermSize=256m -Xss1024K -Xms7G -Xmx7G -XX:+DisableExplicitGC -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Dalfresco.home=/opt/alfresco -Dcom.sun.management.jmxremote -Dsun.security.ssl.allowUnsafeRenegotiation=true -server"
…
‍‍‍‍‍

I didn't change anything in /alfrescoinstall/tomcat/bin/setenv.sh and it has:


JAVA_HOME=/opt/alfresco/java
JRE_HOME=/opt/alfresco/java
JAVA_OPTS="$JAVA_OPTS "
export JAVA_HOME
export JRE_HOME
export JAVA_OPTS
‍‍‍‍‍‍‍‍

Re: Alfresco crashing during working hours

eswbitto — Wed, 24 Feb 2016 16:43:18 GMT

Try putting your parameter here too and then restart alfresco completely. Any java changes (or anything for that matter) require a restart of the service.

JAVA_HOME=/opt/alfrescoinstall/java
JRE_HOME=$JAVA_HOME
JAVA_OPTS="-XX:-DisableExplicitGC -XX:ReservedCodeCacheSize=128m $JAVA_OPTS "
JAVA_OPTS="-XX:MaxPermSize=256M -Xms128M -Xmx4096M $JAVA_OPTS " # java-memory-settings
export JAVA_HOME
export JRE_HOME
export JAVA_OPTS

Re: Alfresco crashing during working hours

douglascrp — Wed, 16 Mar 2016 02:42:40 GMT

It seems you are using Alvex Enterprise.
I'm presuming this based on the log messages.

Why don't you try to contact them?

The error seems to be related with a header extension.
I couldn't find anything odd by looking at the community version's source though.