topic Re: Problems with solr when migratin from Alfresco 5.2 to 6.2 in Alfresco Forum

Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Thu, 26 Aug 2021 03:29:27 GMT

I'm having serious troubles trying to migrate my data from alfresco 5.2 to 6.2.

Everything works but when the reindex is happening, all sort of errors happen, the one that are blocking me this time is:

solr6_1                  | 2021-08-26 03:15:01.896 WARN  (qtp868964689-12) [   ] o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_1x]
solr6_1                  | java.nio.file.NoSuchFileException: /opt/alfresco-search-services/data/archive/index/segments_1x
solr6_1                  |      at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
solr6_1                  |      at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
solr6_1                  |      at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
solr6_1                  |      at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
solr6_1                  |      at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
solr6_1                  |      at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
solr6_1                  |      at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
solr6_1                  |      at java.base/java.nio.file.Files.size(Files.java:2381)
solr6_1                  |      at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
solr6_1                  |      at org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:128)
solr6_1                  |      at org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:615)
solr6_1                  |      at org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:588)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminOperation.getCoreStatus(CoreAdminOperation.java:348)
solr6_1                  |      at org.apache.solr.handler.admin.StatusOp.execute(StatusOp.java:48)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:388)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
solr6_1                  |      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
solr6_1                  |      at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748)
solr6_1                  |      at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729)
solr6_1                  |      at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
solr6_1                  |      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
solr6_1                  |      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
solr6_1                  |      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
solr6_1                  |      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
solr6_1                  |      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
solr6_1                  |      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
solr6_1                  |      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
solr6_1                  |      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
solr6_1                  |      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
solr6_1                  |      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
solr6_1                  |      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
solr6_1                  |      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
solr6_1                  |      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
solr6_1                  |      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
solr6_1                  |      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
solr6_1                  |      at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
solr6_1                  |      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
solr6_1                  |      at org.eclipse.jetty.server.Server.handle(Server.java:539)
solr6_1                  |      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
solr6_1                  |      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
solr6_1                  |      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
solr6_1                  |      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
solr6_1                  |      at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
solr6_1                  |      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
solr6_1                  |      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
solr6_1                  |      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
solr6_1                  |      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
solr6_1                  |      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
solr6_1                  |      at java.base/java.lang.Thread.run(Thread.java:829)

After that error the reindex seems to stop.

Thats a strange error to happen since I didn't brought the index from the old version since it's not compatible.

I'm using the alfresco community docker installer:

https://github.com/Alfresco/alfresco-docker-installer

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Thu, 26 Aug 2021 03:36:10 GMT

Sorry for the double post but now i'm also getting this error several times:

solr6_1                  | 2021-08-26 03:34:31.427 ERROR (org.alfresco.solr.AlfrescoCoreAdminHandler@63a5e46c_Worker-2) [   ] o.a.s.h.RequestHandlerBase java.lang.NullPointerException
solr6_1                  |      at org.apache.solr.update.UpdateLog.lookup(UpdateLog.java:968)
solr6_1                  |      at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:219)
solr6_1                  |      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:296)
solr6_1                  |      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.executeQueryRequest(SolrInformationServer.java:3387)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.getTrackerInitialState(SolrInformationServer.java:1415)
solr6_1                  |      at org.alfresco.solr.tracker.AbstractTracker.getTrackerState(AbstractTracker.java:332)
solr6_1                  |      at org.alfresco.solr.tracker.AbstractTracker.track(AbstractTracker.java:227)
solr6_1                  |      at org.alfresco.solr.tracker.ActivatableTracker.track(ActivatableTracker.java:97)
solr6_1                  |      at org.alfresco.solr.tracker.TrackerJob.execute(TrackerJob.java:60)
solr6_1                  |      at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
solr6_1                  |      at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
solr6_1                  |
solr6_1                  | 2021-08-26 03:34:31.429 ERROR (org.alfresco.solr.AlfrescoCoreAdminHandler@63a5e46c_Worker-2) [   ] o.a.s.SolrInformationServer Got no response from a tracker initial state request.
solr6_1                  | 2021-08-26 03:34:31.437 ERROR (org.alfresco.solr.AlfrescoCoreAdminHandler@63a5e46c_Worker-12) [   ] o.a.s.h.RequestHandlerBase java.lang.NullPointerException
solr6_1                  |      at org.apache.solr.update.UpdateLog.lookup(UpdateLog.java:968)
solr6_1                  |      at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:219)
solr6_1                  |      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:296)
solr6_1                  |      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.executeQueryRequest(SolrInformationServer.java:3387)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.getTrackerInitialState(SolrInformationServer.java:1415)
solr6_1                  |      at org.alfresco.solr.tracker.ShardStatePublisher.getTrackerState(ShardStatePublisher.java:123)
solr6_1                  |      at org.alfresco.solr.tracker.AbstractTracker.track(AbstractTracker.java:227)
solr6_1                  |      at org.alfresco.solr.tracker.TrackerJob.execute(TrackerJob.java:60)
solr6_1                  |      at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
solr6_1                  |      at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
solr6_1                  |

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

angelborroy — Thu, 26 Aug 2021 07:28:24 GMT

How are you deploying Alfresco?

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Thu, 26 Aug 2021 11:18:10 GMT

I'm using the community docker installer: https://github.com/Alfresco/alfresco-docker-installer

And also the alfresco s3 adapter (version 1.2.1)

Regards,

Iohann

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

angelborroy — Thu, 26 Aug 2021 15:13:12 GMT

That should be a problem with Docker Volumes binding for SOLR. Review that section in project documentation in order to be sure that everything is working fine.

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Thu, 26 Aug 2021 18:30:40 GMT

I don't know... solr is able to reach the folder. The index sometimes grows to about 2gb but then it hangs. Looking at Solr Admin it seems to always hang at aroung 130k docs in the index.

I even installed another machine with the same configurations, but without data migration, and it works just fine...

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

angelborroy — Fri, 27 Aug 2021 06:32:18 GMT

How many RAM is assigned to your SOLR Docker service?

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

angelborroy — Fri, 27 Aug 2021 14:58:29 GMT

Did you try increasing RAM for this service?

This may be a cause of your system running consistently but failing from 130k docs.

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Fri, 27 Aug 2021 16:38:20 GMT

I try a few hours ago:

    solr6:
        build:
          context: ./search
          args:
            SEARCH_TAG: $SEARCH_CE_TAG
            SOLR_HOSTNAME: solr6
            ALFRESCO_HOSTNAME: alfresco
            ALFRESCO_COMMS: none
            CROSS_LOCALE: "true"
        mem_limit: 4096m
        environment:
            #Solr needs to know how to register itself with Alfresco
            SOLR_ALFRESCO_HOST: "alfresco"
            SOLR_ALFRESCO_PORT: "8080"
            #Alfresco needs to know how to call solr
            SOLR_SOLR_HOST: "solr6"
            SOLR_SOLR_PORT: "8983"
            #Create the default alfresco and archive cores
            SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
            SOLR_JAVA_MEM: "-Xms3584m -Xmx3584m"
            SOLR_OPTS: "
                -XX:NewSize=2048m
                -XX:MaxNewSize=2048m
            "
        volumes: 
            - ./data/solr-data:/opt/alfresco-search-services/data

Same result... in solr admin, "archive" core shows around 134k documents indexed. "alfresco" core, shows around 9k

just a note: my previous message is missing: i didn't deleted it.

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Fri, 17 Sep 2021 12:21:26 GMT

Solr used 100gb while indexing and took 2 days crashing sometimes, but eventually it commited the whole tlog and now the index is much smaller.

Now it's working fine...

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

wybrand — Mon, 29 Nov 2021 14:39:41 GMT

Hi Johann,

We are having the same problems, so I read you post. But I cannot figure out what exactly you did to solve your problems.

Can you explain??

Best regards,

Wybrand.

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

iohann95 — Tue, 07 Dec 2021 13:33:58 GMT

Hi Wybrand,

Basically I allocated a bunch of ram to Solr service, if I recall I put 6144mb on mem_limit. I also changed Xmx and Xms to 4608mb. I intentionally left some wiggle room because otherwise the docker container kept crashing with error 137.

tlog when not comitted seems to also eat alot of HDD, so mind that also.

After reinxed completed I changed back the configurations. I'm no expert, but that's what worked for me.

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

theDreamer — Sun, 07 Apr 2024 15:38:53 GMT

Hello,

I encountered this issue. Where did adjust the solr memory it is in sorl GUI?

Thank you.