Cannot shutdown with 4.0.d (Linux 64)

[michaelk]

I am not able to shutdown using 4.0.d (no problem with previous versions 4.0.c/b/a).
The shutdown script tries to first stop Tomcat but hangs (and never gets to shutting down postgrsql and openoffice).
It looks like that Tomcat is generating "severe" errors when shutdown is initiated. catalina.out shows lots of errors (see below)
This makes routine maintenance impossible. Does anyone know what could be the problem ?

Michael

……….
Feb 9, 2012 3:46:25 AM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Feb 9, 2012 3:46:26 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/share] created a ThreadLocal with key of type [null] (value [null]) and a value of type [freemarker.template.DefaultObjectWrapper] (value [freemarker.template.DefaultObjectWrapper@5bb1ffad]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Feb 9, 2012 3:46:26 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/share] created a ThreadLocal with key of type [null] (value [org.springframework.extensions.webscripts.processor.FTLTemplateProcessor$NonBlockingObjectWrapper$1@308e5e3]) and a value of type [freemarker.template.DefaultObjectWrapper] (value [freemarker.template.DefaultObjectWrapper@3d2c9ba3]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Feb 9, 2012 3:46:26 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/share] created a ThreadLocal with key of type [null] (value [org.springframework.extensions.webscripts.processor.FTLTemplateProcessor$NonBlockingObjectWrapper$1@3b9e09b5]) and a value of type [freemarker.template.DefaultObjectWrapper] (value [freemarker.template.DefaultObjectWrapper@48f68a7]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Feb 9, 2012 3:46:26 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap

….  lots more like this

SEVERE: The web application [/alfresco] created a ThreadLocal with key of type [null] (value [org.alfresco.repo.tagging.TagScopePropertyMethodInterceptor$1@5fe170c5]) and a value of type [java.lang.Boolean] (value [true]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
log4j:ERROR LogMananger.repositorySelector was null likely due to error in class reloading, using NOPLoggerRepository.
Feb 9, 2012 3:46:43 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/solr] appears to have started a thread named [MultiThreadedHttpConnectionManager cleanup] but has failed to stop it. This is very likely to create a memory leak.
Feb 9, 2012 3:46:43 AM org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8080
Feb 9, 2012 3:46:43 AM org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8443
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[loftux]

There is an alfresco thread

"RMI RenewClean-[204.187.13.213:38436,org.alfresco.util.remote.server.socket.HostConfigurableSocketFactory@0]" daemon prio=10 tid=0x0000000040bd3000 nid=0x6efd in Object.wait() [0x00007ff0b9b4e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on <0x00007ff0eaf50b30> (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
   - locked <0x00007ff0eaf50b30> (a java.lang.ref.ReferenceQueue$Lock)
   at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)
   at java.lang.Thread.run(Thread.java:662)‍‍‍‍‍‍‍‍

[michaelk]

I understand that the thread dump didn't show an Alfresco thread but how does this help in determining what is hanging up the shutdown ?

***edit****

There is an alfresco thread

Oh, I see what you are getting at… yes I see that there is an Alfreso thread in there.
**** end edit *****
If the shutdown has successfully killed the Alfresco threads (presumably what is implied), then something in the Tomcat server configuration as distributed is wrong.

"DestroyJavaVM" prio=10 tid=0x0000000040113000 nid=0x6e9e waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE‍‍

"Condition 0x0000000000000000"  can't be good!

What's odd is that Alfresco can't reproduce this but it is happening in the field to a number of different installation setups.

[jordiv]

It should be fixed for the next versions:
https://issues.alfresco.com/jira/browse/ALF-13169


Cheers,
Jordi.

[jonvargas]

Currently no real problem caused, but the server takes 5 minutes to stop and those SOLR errors still appear.

I am using as a workaround changing the 300 seconds kill timeout in tomcat/scripts/ctl.sh to a lower number, for example, 60 seconds.

[dnallsopp]

I see a similar issue with Alfresco Community 4.0.e, on a fresh install, using the 64-bit installer on an Ubuntu LXC container (Ubuntu LTS 12.04)

Startup takes about 15 seconds. Shutdown takes over 5 minutes:

ubuntu@alfresco4e:/opt/alfresco-4.0.e$ time sudo ./alfresco.sh start
/opt/alfresco-4.0.e/postgresql/scripts/ctl.sh : postgresql  started at port 5432
…
/opt/alfresco-4.0.e/tomcat/scripts/ctl.sh : tomcat started
real   0m16.047s

ubuntu@alfresco4e:/opt/alfresco-4.0.e$ time sudo ./alfresco.sh stop 
…
Killing: 7015
/opt/alfresco-4.0.e/tomcat/scripts/ctl.sh : tomcat stopped
/opt/alfresco-4.0.e/postgresql/scripts/ctl.sh : postgresql stopped
real   5m14.147s
‍‍‍‍‍‍‍‍‍‍‍‍‍

Lots of errors in catalina.out - a few examples:

SEVERE: The web application [/alfresco] appears to have started a thread named [SolrWatcherScheduler_Worker-1] but has failed to stop it. This is very likely to create a memory leak.
…
SEVERE: The web application [/alfresco] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@ec3ae61]) and a value of type [org.alfresco.util.CachingDateFormat] (value [yyyy-MM-dd]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
…
SEVERE: The web application [/solr] appears to have started a thread named [SolrTrackerScheduler_Worker-1] but has failed to stop it. This is very likely to create a memory leak.‍‍‍‍‍

[dwreynoldsjr]

Centos 6 64-bit, Alfresco 4.0e, vanilla install from .bin

Same problem.  Won't shutdown, Catalina SEVERE errors.

[michaelk]

According to https://forums.alfresco.com/en/viewtopic.php?f=8&t=45940 4.0.e is just 4.0.d with new SOLR certificates. Therefore the bug persists in 4.0.e…  We hope the fix is included in 4.0.f 8-)