OOM and MailContentTransformer Error

[alftier]

Hi,

My company has a very old version of alfresco:
Alfresco 2.1.0-482 on JBoss 4.0.4.GA.

Recently we are having OutOfMemory issues that are forcing us to restart the service.

10:30:50,242 ERROR [STDERR] Exception in thread "Sess_FTP1009_192.168.22.213" 
10:30:50,242 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.index.MultiReader.norms(MultiReader.java:173)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.index.FilterIndexReader.norms(FilterIndexReader.java:116)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:69)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.BooleanQuery$BooleanWeight2.scorer(BooleanQuery.java:355)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:130)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:100)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:66)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.Hits.<init>(Hits.java:45)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.Searcher.search(Searcher.java:45)
10:30:50,243 ERROR [STDERR] 	at org.apache.lucene.search.Searcher.search(Searcher.java:37)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:272)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:85)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.filefolder.FileFolderServiceImpl.luceneSearch(FileFolderServiceImpl.java:498)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.filefolder.FileFolderServiceImpl.searchInternal(FileFolderServiceImpl.java:464)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.filefolder.FileFolderServiceImpl.search(FileFolderServiceImpl.java:358)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.filefolder.FileFolderServiceImpl.search(FileFolderServiceImpl.java:343)
10:30:50,243 ERROR [STDERR] 	at sun.reflect.GeneratedMethodAccessor353.invoke(Unknown Source)
10:30:50,243 ERROR [STDERR] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
10:30:50,243 ERROR [STDERR] 	at java.lang.reflect.Method.invoke(Method.java:597)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:281)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.filefolder.TempFileMarkerInterceptor.invoke(TempFileMarkerInterceptor.java:83)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.ml.MLContentInterceptor.invoke(MLContentInterceptor.java:129)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.model.filefolder.MLTranslationInterceptor.invoke(MLTranslationInterceptor.java:210)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
10:30:50,243 ERROR [STDERR] 	at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
10:30:50,243 ERROR [STDERR] 	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:49)
10:30:50,243 ERROR [STDERR] 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)

Analyzing the logs, the only error of the current year started to appear on 08/24/21 and its flooding the log file (I don't know if this is related to the oom)

11:32:02,248 WARN [MailContentTransformer] Unable to extract meta-data from message: Unable to read entire block; 332 bytes read; expected 512 bytes
11:32:02,562 ERROR [JobRunShell] Job DEFAULT.ftsIndexerJobDetail threw an unhandled Exception: 
org.alfresco.service.cmr.repository.ContentIOException: Failed to open stream onto channel: 
accessor: ContentAccessor[ contentUrl=store://2021/11/24/11/32/c2ed587d-4d11-11ec-83d6-07aac04c23f1.bin, mimetype=null, size=0, encoding=UTF-8, locale=es_ES]
at org.alfresco.repo.content.AbstractContentReader.getContentInputStream(AbstractContentReader.java:337)
at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.indexProperty(ADMLuceneIndexerImpl.java:858)
at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocuments(ADMLuceneIndexerImpl.java:542)
at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.updateFullTextSearch(ADMLuceneIndexerImpl.java:1248)
at org.alfresco.repo.search.impl.lucene.fts.FullTextSearchIndexerImpl.index(FullTextSearchIndexerImpl.java:188)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:281)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:210)
at $Proxy59.index(Unknown Source)
at org.alfresco.repo.search.impl.lucene.fts.FTSIndexerJob.execute(FTSIndexerJob.java:52)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)
Caused by: java.lang.UnsupportedOperationException: The content never exists
at org.alfresco.repo.content.EmptyContentReader.getDirectReadableChannel(EmptyContentReader.java:59)
at org.alfresco.repo.content.AbstractContentReader.getReadableChannel(AbstractContentReader.java:226)
at org.alfresco.repo.content.AbstractContentReader.getContentInputStream(AbstractContentReader.java:330)
... 18 more
11:32:02,659 ERROR [ErrorLogger] Job (DEFAULT.ftsIndexerJobDetail threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.service.cmr.repository.ContentIOException: Failed to open stream onto channel: 
accessor: ContentAccessor[ contentUrl=store://2021/11/24/11/32/c2ed587d-4d11-11ec-83d6-07aac04c23f1.bin, mimetype=null, size=0, encoding=UTF-8, locale=es_ES]]
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)
Caused by: org.alfresco.service.cmr.repository.ContentIOException: Failed to open stream onto channel: 
accessor: ContentAccessor[ contentUrl=store://2021/11/24/11/32/c2ed587d-4d11-11ec-83d6-07aac04c23f1.bin, mimetype=null, size=0, encoding=UTF-8, locale=es_ES]
at org.alfresco.repo.content.AbstractContentReader.getContentInputStream(AbstractContentReader.java:337)
at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.indexProperty(ADMLuceneIndexerImpl.java:858)
at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocuments(ADMLuceneIndexerImpl.java:542)
at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.updateFullTextSearch(ADMLuceneIndexerImpl.java:1248)
at org.alfresco.repo.search.impl.lucene.fts.FullTextSearchIndexerImpl.index(FullTextSearchIndexerImpl.java:188)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:281)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:210)
at $Proxy59.index(Unknown Source)
at org.alfresco.repo.search.impl.lucene.fts.FTSIndexerJob.execute(FTSIndexerJob.java:52)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
... 1 more
Caused by: java.lang.UnsupportedOperationException: The content never exists
at org.alfresco.repo.content.EmptyContentReader.getDirectReadableChannel(EmptyContentReader.java:59)
at org.alfresco.repo.content.AbstractContentReader.getReadableChannel(AbstractContentReader.java:226)
at org.alfresco.repo.content.AbstractContentReader.getContentInputStream(AbstractContentReader.java:330)
... 18 more

I have only found one reference to the error :

https://hub.alfresco.com/t5/forum-archive/r%C3%A9solu-erreurs-r%C3%A9currentes-error-org-quartz-core...

It seems to be related to the outlook message transformer, but I do not know if it is safe to disable it, nor how to do it, since the configuration xml is packed in the war:

# grep -ir "transformer.OutlookMsg" *
server/alfresco/tmp/deploy/tmp1069080629325672482alfresco-exp.war/WEB-INF/classes/alfresco/content-services-context.xml:   <bean id="transformer.OutlookMsg"

Can the MailContentTransformer error be related to the OOM?

I appreciate any help, we are trying to connect jboss 4 to visualvm to try to get the graphs out of heap memory, but it is an old system and we are having a hard time finding documentation on how to manage it.

We are also having trouble pulling a heapdump (I'm a sysadmin, I don't understand too much java).

Thanks in advance, best regards

PD: SOrry for my english