Sometimes we cannot login

[weissenboeck]

we use 201602
sometimes we find follwoing lines in the stdout.log
always 04:00
if this happens then we cannot login any more

2016-05-12 04:00:01,126  ERROR [solr.servlet.SolrDispatchFilter] [http-bio-8743-exec-7] nullrg.apache.solr.common.SolrException: SolrCore 'archive' is not available due to init failure: Error opening new searcher
   at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745)
   at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:299)
   at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
   at org.alfresco.web.scripts.servlet.X509ServletFilterBase.doFilter(X509ServletFilterBase.java:144)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
   at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
   at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
   at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
   at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   at org.apache.solr.core.SolrCore.<init>(SolrCore.java:881)
   at org.apache.solr.core.SolrCore.<init>(SolrCore.java:654)
   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
   at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
   at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
   at java.util.concurrent.FutureTask.run(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   … 1 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1574)
   at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1686)
   at org.apache.solr.core.SolrCore.<init>(SolrCore.java:853)
   … 8 more
Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in NRTCachingDirectory(MMapDirectory@C:\alfresco-community\alf_data\solr4\index\archive\SpacesStore\index lockFactory=NativeFSLockFactory@C:\alfresco-community\alf_data\solr4\index\archive\SpacesStore\index; maxCacheMB=48.0 maxMergeSizeMB=4.0): files: [write.lock]
   at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:864)
   at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:752)
   at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:450)
   at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:792)
   at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
   at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
   at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:279)
   at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:111)
   at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1537)
   … 10 more

2016-05-12 04:00:01,188  ERROR [quartz.core.JobRunShell] [SearchScheduler_Worker-4] Job DEFAULT.search.archiveCoreBackupJobDetail threw an unhandled Exception:
org.alfresco.error.AlfrescoRuntimeException: 041212014 Backup for core archive failed ….
   at org.alfresco.repo.search.impl.solr.SolrBackupClient.executeImpl(SolrBackupClient.java:196)
   at org.alfresco.repo.search.impl.solr.SolrBackupClient.execute(SolrBackupClient.java:133)
   at org.alfresco.repo.search.impl.solr.SolrBackupJob.execute(SolrBackupJob.java:58)
   at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
   at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:563)
Caused by: org.apache.solr.client.solrj.SolrServerException: Error executing query
   at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95)
   at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118)
   at org.alfresco.repo.solr.SOLRAdminClient.query(SOLRAdminClient.java:196)
   at org.alfresco.repo.search.impl.solr.SolrBackupClient.executeImpl(SolrBackupClient.java:185)
   … 4 more
Caused by: org.apache.solr.common.SolrException: Internal Server Error

Internal Server Error

request: https://localhost:8743/solr4/archive/replication?qt=/archive/replication&command=backup&location=\\s...
   at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:435)
   at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
   at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
   … 7 more
2016-05-12 04:00:01,235  ERROR [quartz.core.ErrorLogger] [SearchScheduler_Worker-4] Job (DEFAULT.search.archiveCoreBackupJobDetail threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.error.AlfrescoRuntimeException: 041212014 Backup for core archive failed …. ]
   at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
   at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:563)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 041212014 Backup for core archive failed ….
   at org.alfresco.repo.search.impl.solr.SolrBackupClient.executeImpl(SolrBackupClient.java:196)
   at org.alfresco.repo.search.impl.solr.SolrBackupClient.execute(SolrBackupClient.java:133)
   at org.alfresco.repo.search.impl.solr.SolrBackupJob.execute(SolrBackupJob.java:58)
   at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
   … 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Error executing query
   at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95)
   at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118)
   at org.alfresco.repo.solr.SOLRAdminClient.query(SOLRAdminClient.java:196)
   at org.alfresco.repo.search.impl.solr.SolrBackupClient.executeImpl(SolrBackupClient.java:185)
   … 4 more
Caused by: org.apache.solr.common.SolrException: Internal Server Error

Internal Server Error

[hiten_rastogi1]

Hi Weissenboeck,

I faced this issue today. Please let me know if you were able to get to the root cause and solution that you tried.