JVM crashes with Sun JVM 1.5.0_6

[kevinr]

If you are running Sun JVM 1.5.0_6 (Windows or Linux) you may see the Alfresco server crash with the following report:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0x40520cd1, pid=31769, tid=2309221296 # # Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode) # Problematic frame:
# V  [libjvm.so+0x3c9cd1]
#

—————  T H R E A D  —————

Current thread (0x89602788):  JavaThread "CompilerThread1" daemon [_thread_in_native, id=31780]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000000

Registers:
EAX=0x00000000, EBX=0x407027f0, ECX=0x89a3cec0, EDX=0x89dafc44 ESP=0x89a3ce34, EBP=0x89a3ce58, ESI=0x89db336c, EDI=0x89a3cec0 EIP=0x40520cd1, CR2=0x00000000, EFLAGS=0x00010292

Top of Stack: (sp=0x89a3ce34)
0x89a3ce34:   89a3d470 89a3d480 40520c16 8a8f8ca8
0x89a3ce44:   8a8f8cc8 89a3ce78 407027f0 89db336c
0x89a3ce54:   89db3328 89a3ce88 40520d75 89db3a60
0x89a3ce64:   89dafc44 89db3a60 89a3cec0 89db3ac4
0x89a3ce74:   89db4168 89db3328 407027f0 89db3a60
0x89a3ce84:   00000000 89a3cee8 40521258 89db3a60
0x89a3ce94:   89db336c 89db3a60 89a3cec0 ffffffff
0x89a3cea4:   89db4704 89a3cee8 405a5bf1 89db4168 

Instructions: (pc=0x40520cd1)
0x40520cc1:   10 eb de 31 c0 eb 8e 8b 42 04 83 ec 0c 8b 40 08
0x40520cd1:   8b 08 50 ff 51 28 83 c4 10 e9 48 ff ff ff 90 55 

Stack: [0x899be000,0x89a3f000),  sp=0x89a3ce34,  free space=507k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V  [libjvm.so+0x3c9cd1] V  [libjvm.so+0x3c9d75] V  [libjvm.so+0x3ca258] V  [libjvm.so+0x4511fd] V  [libjvm.so+0x450e90] V  [libjvm.so+0x1a11c0] V  [libjvm.so+0x19e432] V  [libjvm.so+0x1474b3] V  [libjvm.so+0x1a6929] V  [libjvm.so+0x1a6281] V  [libjvm.so+0x4c8366] V  [libjvm.so+0x4c2ba3] V  [libjvm.so+0x424338] C  [libpthread.so.0+0x5cb7]

[b]Current CompileTask:
opto:2497      org.apache.lucene.index.IndexReader$1.doBody()Ljava/lang/Object; (99 bytes)[/b]
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This has been identified as a JVM compiler (HotSpot) bug in that particular version of the JVM.

You can either:
1. Revert to Sun JVM 1.5.0_5 (which does not appear to have the problem)
2. Remove the method from JIT compliation:

This is done by creating a ".hotspot_compiler" file where java is called.
See the attached file which excludes the offending method.
For tomcat, this file would go in the TOMCAT_HOME/bin directory.
For use from Alfresco source, it would go in the project directory: e.g. \alfresco-src\root\projects\repository

To confirm this file is being picked up:
You should see the following as the JVM start up:

CompilerOracle: exclude org/apache/lucene/index/IndexReader$1 doBody‍

When an attempt to compile this method is made, it will be rejected and you will see:

### Excluding compile:  org.apache.lucene.index.IndexReader$1::doBody

If you want to see all compilation progress you can use  XX:+PrintCompilation

The file should contain the following line:

exclude  org/apache/lucene/index/IndexReader$1 doBody‍

It is possible the JVMs differ in how they find this file:

http://sunsolve.sun.com/searchproxy/document.do?assetkey=1-9-73893-1

[sysarch]

It is not capturing the file. I don't know how to resolve the issue, it is the stock windows download off the web site so its very wierd it is not picking it up properly.  The Sun link was also dead.

[andy]

Hi

Which OS/JVM are you using?
Where are you putting the file?

It all works for me with 1.5.0_06 on windows.
I am sure we have also tested this fix for linux.

Regards

Andy

[elam]

Hi Kevin,

I'm using the 1.5_06-b05( newest) and the Alfresco JBOSS distribution and I see that the .hotspot_compiler file is already in alfresco/jboss/bin and I'm getting the crash.  The link for the sun bug report is also dead.

Cheers,
-Elam

[kevinr]

Hi,

FYI with the latest bundles we are using:
-XX:CompileCommand=exclude,org/apache/lucene/index/IndexReader$1,doBody
JAVA_OPTS argument instead of the .hotspot_compiler file to eliminate these issues.

Thanks,

Kevin

[osu]

Hi Community,

I suppose people around here have been trying other JVMs than the bundled version 1.5.0_06?

I experienced crashes with CIFS/FTP uploads as well as with a modified version of the Webservices Import sample code with the above JVM. With JDK 1.5.0_08 as a replacement of the bundled JVM I haven't been able to reproduce the problem. Looking good so far, but still testing.

Regards,
Oliver.

[kevinr]

We found a couple of further issues with the 1.5.0_06 JVM - it definitely has a few bugs! The following was required in .hotspot_compiler file to eliminate all HotSpot issues:

exclude  org/apache/lucene/index/IndexReader$1 doBody
exclude  org/alfresco/repo/search/impl/lucene/index/IndexInfo$Merger mergeIndexes
exclude  org/alfresco/repo/search/impl/lucene/index/IndexInfo$Merger mergeDeletions

Hopefully the 1.5.0_08 fixes those problems!

Thanks for letting us know,

Kevin

[geert_bontridde]

I seem to have the same problem, but the strange thing is I'm using JVM1.5.0_05-b05

In My log I get:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0xb7a08c41, pid=1742, tid=2386381744
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_05-b05 mixed mode)
# Problematic frame:
# V  [libjvm.so+0x39ec41]
#
# An error report file with more information is saved as hs_err_pid1742.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
CompilerOracle: exclude org/apache/lucene/index/IndexReaderstart doBody
Sep 15, 2006 1:02:21 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
…
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So is reverting to this version really a solution?

[kevinr]

I said 1.5.0_08 not 1.5.0_05. And yes I can confirm that the 3 hotspot bugs are indeed fixed in 1.5.0_08.

Thanks,

Kevin

[geert_bontridde]

Yes, that's what you said in your post of sep 6, but in the first post of this issue (jan 26) you said:

You can either:
1. Revert to Sun JVM 1.5.0_5 (which does not appear to have the problem)
2. Remove the method from JIT compliation

…and since I'm currently using JVM 1.5.0_5 I wonder why I seem to have the same problem…?

Thanks,