topic Alfresco Lucene Indexing in Alfresco Archive

Alfresco Lucene Indexing

alfsender — Thu, 21 Nov 2013 17:30:59 GMT

Hi All,We are having an Alfresco repository which has more then 2 million records. The records are categorized in below folder structure.Main_Folder Sub_folder_1 folder_1 (5,000 Docs) folder_2 (5,000 Docs) folder_3 (5,000 Docs) … … folder_1000 (5,000 Docs)

Re: Alfresco Lucene Indexing

rjohnson — Fri, 22 Nov 2013 00:28:37 GMT

I remember having this issue myself when doing an index rebuild on a 4.0a system with only 150,000 documents. I found a very good note on it in the forums (somewhere). Now, I did not keep a copy of it verbatim, nor the link, but looking in my alfresco-global.properties I think the solution for me was to:-

Stop Alfresco.
Add the line lucene.indexer.batchSize=1000 to alfresco-global.properties
Set index.recovery.mode=FULL
Restart Alfresco
Let the indexes build
Stop Alfresco
Comment out lucene.indexer.batchSize=1000
Set index.recovery.mode=AUTO
Restart Alfresco.

If you want to avoid a full rebuild you could try stopping Alfresco, just adding the

lucene.indexer.batchSize=1000

restaring Alfresco and letting it run until it is caught up. That may or may not work because the post was specific about setting it for a rebuild then "unsetting" it for normal running.

Please let me know how you get on.

Re: Alfresco Lucene Indexing

alfsender — Fri, 22 Nov 2013 01:59:46 GMT

Thank you Sir, for you response.

Actually we have two servers in clustered, so one node this what we have done and restarted server in for FULL indexing.

But reason this is happening in out case is because when we add groups (through alfresco explorer) to space "sub_folder_1" it creates a single transaction to add groups to all sub documents, and as there are million records it creates single transaction for 14 million documents. So when we do a restart it tries to index last transaction i.e. the one with 14 millions records, and after some time it give above error with cache FULL.

Is there a way in which we can tell to Alfresco that do index all transaction except the one with 14 million records, . Or can we just delete that transaction entry from Transaction table in Alfresco. If we do that what are the other table that might get affected because of that and what are the other places where this might affect ?

Thank you.

Re: Alfresco Lucene Indexing

abarisone — Fri, 22 Nov 2013 07:26:00 GMT

Hi,
The cache full is rather a warning than an error.
I did a reindex on the other day with an almost empty system and I got the same message.
The full reindex took about half an hour to run but everything went fine.
Avoid directly modifying the database: as mentioned here http://forums.alfresco.com/forum/developer-discussions/repository-services/increase-cache-contextxml-or-ehcache-customxml transaction cache default values are fine.
If you happen to have a cluster configuration, remember you can always perform a full hot reindex http://docs.alfresco.com/4.2/index.jsp?topic=%2Fcom.alfresco.enterprise.doc%2Ftasks%2Fhot-reindex.html
Also here http://alch3mi5t.blogspot.it/2013/02/lucene-performance-during-index.html you can find some tips on Lucene performance optimization

Regards,
Andrea