topic Re: Upgrade from 4.0d using Lucene to 4.2e using SOLR in Alfresco Archive

Upgrade from 4.0d using Lucene to 4.2e using SOLR

ewan_ritchie — Thu, 25 Sep 2014 15:05:55 GMT

I have just updated from 4.0d with Lucene indexing to 4.2e and have enabled and configured SOLR. Our environment is Windows Server 2008 R2.Everything is working as expected except when searching (advanced search on a custom object model) using more than one filter criteria the results returned (man

Re: Upgrade from 4.0d using Lucene to 4.2e using SOLR

mrogers — Thu, 25 Sep 2014 17:02:17 GMT

Its a feature of 5.0.

https://wiki.alfresco.com/wiki/Alfresco_Community_5.0

Re: Upgrade from 4.0d using Lucene to 4.2e using SOLR

ewan_ritchie — Fri, 26 Sep 2014 07:11:38 GMT

Thanks for the reply.

I read the section about Search Improvements in Alfresco 5.0 and it says "The default search operator for a multi-term search has changed from OR to AND". The term "default" says to me that it is configurable. Can you please confirm this? If it is configurable in 5.0 is it configurable in 4.2e?

What I am looking for how to change the default search operator for a multi-term search in Alfresco 4.2e using SOLR . Could you help my by pointing to the configuration option or provide me details on how to achieve this?

If this is not possible in Alfresco 4.2e using SOLR is it an option for us to revert back to using Lucene indexing? I ask this because I had previously tried to enable Lucene indexing on 4.2e and could not get the system to index at all and it also suffered from really long start times for the webapps running under Tomcat (the system could take 1.5 hours to start which we couldn't find a solution for).

Thanks for your assistance.

Regards
Ewan

Re: Upgrade from 4.0d using Lucene to 4.2e using SOLR

ewan_ritchie — Wed, 01 Oct 2014 08:07:00 GMT

I am grateful to know that that feature is now available in Alfresco 5 but since I have just upgraded to 4.2e (decision made by management prior to Alfresco 5 being released) I am looking for a solution for 4.2e.

I have verified there is an issue by writing some CMIS queries and I get the results I am expecting. A generalized version of the queries I used are below and show where I believe the issue lies:

When searching for documents of a custom type in the Advanced Search screen and entering the following filters the search returns 11 results (which I am told is correct):

Title (free text): OVERHEAD

Site Name (list): REFINERY

Area Code (list): XX45

Discipline (list): ELECTRICAL

When we add in the following filter the search results and it returned more than 1000 documents rather than the 5 documents expected:

Drawing Code (list): EL32 TECHNICAL DOCUMENTS

To verify what to expect from each search we used the following queries:
<CODE>SELECT ed.*,dc.*,t.* FROM cust:engDoc AS ed JOIN cust:documentControl AS dc ON ed.cmisbjectId = dc.cmisbjectId JOIN cm:titled AS t ON ed.cmisbjectId = t.cmisbjectId WHERE t.cm:title LIKE '%OVERHEAD%' AND ed.custiteName = 'REFINERY' AND ed.cust:AreaCode = 'XX45' AND dc.custiscipline = 'ELECTRICAL'</CODE>
Returned 11 results as expected.

<CODE>SELECT ed.*,dc.*,t.* FROM cust:engDoc AS ed JOIN cust:documentControl AS dc ON ed.cmisbjectId = dc.cmisbjectId JOIN cm:titled AS t ON ed.cmisbjectId = t.cmisbjectId WHERE t.cm:title LIKE '%OVERHEAD%' AND ed.custiteName = 'REFINERY' AND ed.cust:AreaCode = 'XX45' AND dc.custiscipline = 'ELECTRICAL' AND ed.custrawingCode = 'EL32 TECHNICAL DOCUMENTS'</CODE>
Returned 5 results as expected.

We have checked and there is mention that in Alfresco 4.2 SOLR uses the OR operator on fields that contain multiple values (i.e. there are spaces in the search term…like "EL32 TECHNICAL DOCUMENTS"). This would mean that the equivalent CMIS search would be something like:
<CODE>SELECT ed.*,dc.*,t.* FROM cust:engDoc AS ed JOIN cust:documentControl AS dc ON ed.cmisbjectId = dc.cmisbjectId JOIN cm:titled AS t ON ed.cmisbjectId = t.cmisbjectId WHERE t.cm:title LIKE '%OVERHEAD%' AND ed.custiteName = 'REFINERY' AND ed.cust:AreaCode = 'XX45' AND dc.custiscipline = 'ELECTRICAL' AND ed.custrawingCode LIKE '%EL32%' OR ed.custrawingCode LIKE '%TECHNICAL%' OR ed.custrawingCode LIKE '%DOCUMENTS%'</CODE>
Returned 18064 results

If I change the OR operators to AND operators the query becomes:
<CODE>SELECT ed.*,dc.*,t.* FROM cust:engDoc AS ed JOIN cust:documentControl AS dc ON ed.cmisbjectId = dc.cmisbjectId JOIN cm:titled AS t ON ed.cmisbjectId = t.cmisbjectId WHERE t.cm:title LIKE '%OVERHEAD%' AND ed.custiteName = 'REFINERY' AND ed.cust:AreaCode = 'XX45' AND dc.custiscipline = 'ELECTRICAL' AND ed.custrawingCode LIKE '%EL32%' AND ed.custrawingCode LIKE '%TECHNICAL%' AND ed.custrawingCode LIKE '%DOCUMENTS%'</CODE>
Returned 5 results

The only solution I can think of at the moment is to update all the lists that have spaces in individual items and replace the spaces with another character (for example an underscore) and then update the meta-data for all content to reflect this (which feels like using a sledgehammer to crack a nut as we have nearly 1 million documents in Alfresco and the update will take a lot of effort and time).

Any help with this issue would be appreciated.

Thanks
Ewan

Re: Upgrade from 4.0d using Lucene to 4.2e using SOLR

ewan_ritchie — Wed, 01 Oct 2014 14:51:53 GMT

I found a really nice plugin called AlfrescoShareANDSearch that seems to mitigate the issues I have seen.

Hopefully it will help other people who have the same issue.

Regards
Ewan