Hi All,
We are running Alfresco community version 4.0 on Linux environment.
Since last few days we are regularly getting too many open files error in alfresco.log file.(snapshot of log file is pasted in the message below)
We tried tuning tomcat "nofile" parameter and maximum possible 65536
We suspect that it might be able to some customization but not able to pin point to any so far
Any pointers/suggestions to analyze issue will be highly appreciated as we our alfresco is getting filled due to this error
Also if you can suggest on following
1. How can we identify which files are being open?
2. Can we prevent the reindexing which might be causing this error?
12:10:55,019 ERROR [org.quartz.core.JobRunShell] Job DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an unhandled Exception:
org.alfresco.error.AlfrescoRuntimeException: 0017151442 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151441 Failed to flush index
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/4ba77f5d-ce9c-4bc1-b162-2fb05fb9d1de/segments_1 (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
... 14 more
12:10:55,019 ERROR [org.quartz.core.ErrorLogger] Job (DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.error.AlfrescoRuntimeException: 0017151442 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker]
at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 0017151442 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
... 1 more
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151441 Failed to flush index
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/4ba77f5d-ce9c-4bc1-b162-2fb05fb9d1de/segments_1 (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
... 14 more
12:10:55,692 ERROR [org.alfresco.repo.search.impl.lucene.index.IndexInfo] Failed building filter reader beneath d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a
java.io.FileNotFoundException: /CMSIndex/lucene-indexes/workspace/SpacesStore/d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a/IndexInfoDeletions (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeletions(IndexInfo.java:1098)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.createMainIndexReader(IndexInfo.java:2099)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(IndexInfo.java:1227)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:168)
at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:287)
at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:44)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:146)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy34.query(Unknown Source)
at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:397)
at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:391)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:390)
at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:387)
at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:508)
at org.alfresco.repo.web.scripts.RepoStore.getDocumentPaths(RepoStore.java:386)
at org.springframework.extensions.webscripts.DeclarativeRegistry.initWebScripts(DeclarativeRegistry.java:223)
at org.springframework.extensions.webscripts.DeclarativeRegistry.reset(DeclarativeRegistry.java:178)
at org.alfresco.repo.web.scripts.RepositoryContainer.getRegistry(RepositoryContainer.java:496)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:143)
at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
12:10:55,693 ERROR [org.alfresco.repo.search.impl.lucene.AbstractLuceneBase] Error
java.io.FileNotFoundException: /CMSIndex/lucene-indexes/workspace/SpacesStore/d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a/IndexInfoDeletions (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeletions(IndexInfo.java:1098)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.createMainIndexReader(IndexInfo.java:2099)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(IndexInfo.java:1227)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:168)
at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:287)
at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:44)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:146)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy34.query(Unknown Source)
at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:397)
at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:391)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:390)
at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:387)
at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:508)
at org.alfresco.repo.web.scripts.RepoStore.getDocumentPaths(RepoStore.java:386)
at org.springframework.extensions.webscripts.DeclarativeRegistry.initWebScripts(DeclarativeRegistry.java:223)
at org.springframework.extensions.webscripts.DeclarativeRegistry.reset(DeclarativeRegistry.java:178)
at org.alfresco.repo.web.scripts.RepositoryContainer.getRegistry(RepositoryContainer.java:496)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:143)
at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
12:10:55,693 ERROR [org.springframework.extensions.webscripts.AbstractRuntime] Exception from executeScript - redirecting to status template error: 0017151443 Failed to open IndexSarcher for /CMSIndex/lucene-indexes/workspace/SpacesStore/
org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151443 Failed to open IndexSarcher for /CMSIndex/lucene-indexes/workspace/SpacesStore/
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:184)
at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:287)
at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:44)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:146)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy34.query(Unknown Source)
at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:397)
at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:391)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:390)
at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:387)
at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:508)
at org.alfresco.repo.web.scripts.RepoStore.getDocumentPaths(RepoStore.java:386)
at org.springframework.extensions.webscripts.DeclarativeRegistry.initWebScripts(DeclarativeRegistry.java:223)
at org.springframework.extensions.webscripts.DeclarativeRegistry.reset(DeclarativeRegistry.java:178)
at org.alfresco.repo.web.scripts.RepositoryContainer.getRegistry(RepositoryContainer.java:496)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:143)
at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/workspace/SpacesStore/d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a/IndexInfoDeletions (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeletions(IndexInfo.java:1098)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.createMainIndexReader(IndexInfo.java:2099)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(IndexInfo.java:1227)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:168)
... 48 more
12:11:00,023 ERROR [org.quartz.core.JobRunShell] Job DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an unhandled Exception:
org.alfresco.error.AlfrescoRuntimeException: 0017151445 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151444 Failed to flush index
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/75ca677a-2c28-479e-abdd-766087d8ae77/segments_1 (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
... 14 more
12:11:00,025 ERROR [org.quartz.core.ErrorLogger] Job (DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.error.AlfrescoRuntimeException: 0017151445 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker]
at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 0017151445 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
... 1 more
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151444 Failed to flush index
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/75ca677a-2c28-479e-abdd-766087d8ae77/segments_1 (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
... 14 more
