topic 201702 solr6 not indexing all documents in Alfresco Forum

201702 solr6 not indexing all documents

xarope — Mon, 10 Apr 2017 04:27:02 GMT

I've just upgraded my previous alfresco 50d instance to 201702, and also switched to solr6.Left the new instance indexing over the weekend, but on morning morning when I checked (after almost 36 hours), it had only indexed perhaps 12% of documents, and from previous experience: (1) the CPU usage is

Re: 201702 solr6 not indexing all documents

xarope — Tue, 11 Apr 2017 04:59:56 GMT

I also ran a solr6 report (i.e. http://<solrserver>:8983/solr/admin/cores?action=REPORT&wt=xml ), here is the alfresco core section:

<long name="DB acl transaction count">8229</long>
<long name="Count of duplicated acl transactions in the index">0</long>
<long name="Count of acl transactions in the index but not the DB">0</long>
<long name="Count of missing acl transactions from the Index">0</long>
<long name="Index acl transaction count">8231</long>
<long name="Index unique acl transaction count">8231</long>
<long name="Last indexed change set commit time">1491820680654</long>
<str name="Last indexed change set commit date">2017-04-10T18:38:00</str>
<long name="Last changeset id before holes">-1</long>
<long name="DB transaction count">1372926</long>
<long name="Count of duplicated transactions in the index">0</long>
<long name="Count of transactions in the index but not the DB">134</long>
<long name="First transaction in the index but not the DB">815448</long>
<long name="Count of missing transactions from the Index">1054</long>
<long name="First transaction missing from the Index">4920501</long>
<long name="Index transaction count">179183</long>
<long name="Index unique transaction count">179183</long>
<long name="Index node count">243985</long>
<long name="Count of duplicate nodes in the index">71</long>
<long name="First duplicate node id in the index">4772300</long>
<long name="Index error count">4</long>
<long name="Count of duplicate error docs in the index">0</long>
<long name="Index unindexed count">7236</long>
<long name="Count of duplicate unindexed docs in the index">0</long>
<long name="Last indexed transaction commit time">1491877238167</long>
<str name="Last indexed transaction commit date">2017-04-11T10:20:38</str>

Based on the documentation in Unindexed Solr Transactions | Alfresco Documentation, seems like there are index errors, e.g. transaction 815448 ("First transaction in the index but not the DB") and 4920501 ("First transaction missing from the Index"). More digging around solr6 documentation to see if I can understand why!

Re: 201702 solr6 not indexing all documents

xarope — Wed, 12 Apr 2017 06:48:31 GMT

Since I couldn't figure this out, I snapshot'd and created a new database and alfresco+solr instance, and installed solr4. In less than 24 hours, solr4 has indexed 1259637 documents in the alfresco core, and 1158783 documents in the archive core.

If I use a psql statement (I'm using postgresql) to get the number of "content" objects, "SELECT count(*) FROM alf_node AS a, alf_qname AS q WHERE a.type_qname_id=q.id AND q.local_name='content';", I get 1443451. So that's pretty close. But I still can't understand why solr6 is stuck, now more than 4 days, with the same ~400K of documents (and not 1.4M).

Looks like I will be switching back to solr4 over this weekend. I hope others have better luck with solr6.