topic Index Recovery (FULL) Hangs in Alfresco Archive https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234913#M188043 <HTML><HEAD></HEAD><BODY><SPAN>Hi all,</SPAN><BR /><BR /><SPAN>I'm trying to rebuild indexes on&nbsp; Alfresco Community 3.3.0 (2765) schema 4009.</SPAN><BR /><SPAN>Everything goes well till 90%: after that the application hangs and nothing else is written to the log file.</SPAN><BR /><BR /><PRE class="language-none line-numbers"><CODE>12:00:18,314&nbsp; DEBUG [java.sql.PreparedStatement] {pstm-100481} Parameters: [5]<BR />12:00:18,314&nbsp; DEBUG [java.sql.PreparedStatement] {pstm-100481} Types: [java.lang.Long]<BR />12:00:18,314&nbsp; DEBUG [java.sql.ResultSet] {rset-100482} ResultSet<BR />12:00:18,314&nbsp; DEBUG [java.sql.ResultSet] {rset-100482} Header: [id, version, uri]<BR />12:00:18,314&nbsp; DEBUG [java.sql.ResultSet] {rset-100482} Result: [5, 0, <A href="http://www.alfresco.org/model/versionstore/2.0" rel="nofollow noopener noreferrer">http://www.alfresco.org/model/versionstore/2.0</A>]<BR />12:00:21,933&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;10 % complete.<BR />12:00:24,040&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;20 % complete.<BR />12:00:26,091&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;30 % complete.<BR />12:00:27,668&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;40 % complete.<BR />12:00:29,649&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;50 % complete.<BR />12:00:31,229&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;60 % complete.<BR />12:00:33,066&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;70 % complete.<BR />12:00:34,800&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;80 % complete.<BR />12:00:36,618&nbsp; INFO&nbsp; [node.index.FullIndexRecoveryComponent] &nbsp;&nbsp;&nbsp;90 % complete.<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN></CODE></PRE><BR /><SPAN>Monitoring java process with jvisualvm I can see that there is a very slow heap usage and one CPU (we have 8 cores) is used costantly.</SPAN><BR /><SPAN>Making a thread dump I can see that there is one indexthread runnable:</SPAN><BR /><BR /><PRE class="language-none line-numbers"><CODE>"indexTrackerThread10" daemon prio=10 tid=0x00002aab48c51800 nid=0x6ec5 runnable [0x0000000045d42000]<BR />&nbsp;&nbsp; java.lang.Thread.State: RUNNABLE<BR />&nbsp;&nbsp;&nbsp;at java.lang.String.intern(Native Method)<BR />&nbsp;&nbsp;&nbsp;at org.apache.lucene.index.Term.&lt;init&gt;(Term.java:50)<BR />&nbsp;&nbsp;&nbsp;at org.apache.lucene.index.Term.&lt;init&gt;(Term.java:36)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.deleteReference(AbstractLuceneIndexerImpl.java:204)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.deleteImpl(AbstractLuceneIndexerImpl.java:600)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:785)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.doPrepare(ADMLuceneIndexerImpl.java:1665)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.prepare(AbstractLuceneIndexerImpl.java:466)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerAndSearcherFactory.prepare(AbstractLuceneIndexerAndSearcherFactory.java:804)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.beforeCommit(AlfrescoTransactionSupport.java:689)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.TransactionSynchronizationUtils.triggerBeforeCommit(TransactionSynchronizationUtils.java:95)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerBeforeCommit(AbstractPlatformTransactionManager.java:927)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:737)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:394)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:472)<BR />&nbsp;&nbsp;&nbsp;- locked &lt;0x00002aaacd631940&gt; (a org.alfresco.util.transaction.SpringAwareUserTransaction)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:343)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.run(AbstractReindexComponent.java:874)<BR />&nbsp;&nbsp;&nbsp;at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)<BR />&nbsp;&nbsp;&nbsp;at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)<BR />&nbsp;&nbsp;&nbsp;at java.lang.Thread.run(Thread.java:619)<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN></CODE></PRE><BR /><SPAN>and all the others are in TIME_WAITED, like this one:</SPAN><BR /><BR /><PRE class="language-none line-numbers"><CODE>"indexTrackerThread11" daemon prio=10 tid=0x00002aab49209000 nid=0x6ed1 in Object.wait() [0x0000000045e43000]<BR />&nbsp;&nbsp; java.lang.Thread.State: TIMED_WAITING (on object monitor)<BR />&nbsp;&nbsp;&nbsp;at java.lang.Object.wait(Native Method)<BR />&nbsp;&nbsp;&nbsp;- waiting on &lt;0x00002aaacd6357f0&gt; (a org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.waitForHeadOfQueue(AbstractReindexComponent.java:825)<BR />&nbsp;&nbsp;&nbsp;- locked &lt;0x00002aaacd6357f0&gt; (a org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.handleQueue(AbstractReindexComponent.java:987)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.afterCommit(AbstractReindexComponent.java:956)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.afterCompletion(AlfrescoTransactionSupport.java:808)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:168)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:996)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:971)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:799)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)<BR />&nbsp;&nbsp;&nbsp;at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:394)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:472)<BR />&nbsp;&nbsp;&nbsp;- locked &lt;0x00002aaacd635940&gt; (a org.alfresco.util.transaction.SpringAwareUserTransaction)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:343)<BR />&nbsp;&nbsp;&nbsp;at org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.run(AbstractReindexComponent.java:874)<BR />&nbsp;&nbsp;&nbsp;at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)<BR />&nbsp;&nbsp;&nbsp;at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)<BR />&nbsp;&nbsp;&nbsp;at java.lang.Thread.run(Thread.java:619)<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN></CODE></PRE><BR /><SPAN>Please, can someone help ?</SPAN><BR /><SPAN>If you need all the trhead dump, off course, I can upload it.</SPAN><BR /><SPAN>Thanks in advance</SPAN></BODY></HTML> Mon, 31 May 2010 10:27:08 GMT scegliau 2010-05-31T10:27:08Z Index Recovery (FULL) Hangs https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234913#M188043 Hi all,I'm trying to rebuild indexes on&nbsp; Alfresco Community 3.3.0 (2765) schema 4009.Everything goes well till 90%: after that the application hangs and nothing else is written to the log file.12:00:18,314&nbsp; DEBUG [java.sql.PreparedStatement] {pstm-100481} Parameters: [5]12:00:18,314&nbsp; DEBUG [java.sql Mon, 31 May 2010 10:27:08 GMT https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234913#M188043 scegliau 2010-05-31T10:27:08Z Re: Index Recovery (FULL) Hangs https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234914#M188044 <HTML><HEAD></HEAD><BODY><SPAN>Did you ever resolve this issue? I'm in the midst of it now.</SPAN></BODY></HTML> Fri, 02 Mar 2012 15:39:47 GMT https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234914#M188044 nadaoneal 2012-03-02T15:39:47Z Re: Index Recovery (FULL) Hangs https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234915#M188045 <HTML><HEAD></HEAD><BODY><SPAN>Stop Alfresco, you may have to to kill the process.</SPAN><BR /><BR /><SPAN>In alfresco-global.propertiesSet set the index.recovery.mode to FULL and add the following line </SPAN><BR /><PRE class="language-none line-numbers"><CODE>lucene.indexer.batchSize=1000<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN></CODE></PRE><SPAN>Remove the alf_data/lucene-indexes folder</SPAN><BR /><BR /><SPAN>Start Alfresco</SPAN><BR /><BR /><SPAN>Once up, stop Alfresco</SPAN><BR /><BR /><SPAN>Change the recovery mode back to VALIDATE and remove the batchSize line.</SPAN><BR /><BR /><SPAN>See this post </SPAN><A href="https://forums.alfresco.com/en/viewtopic.php?t=10984" rel="nofollow noopener noreferrer">https://forums.alfresco.com/en/viewtopic.php?t=10984</A></BODY></HTML> Wed, 21 Mar 2012 10:28:48 GMT https://connect.hyland.com/t5/alfresco-archive/index-recovery-full-hangs/m-p/234915#M188045 lynnders 2012-03-21T10:28:48Z