topic Index Tip Recovery For Large Repositories in Alfresco Archive

Index Tip Recovery For Large Repositories

callermd — Tue, 12 Feb 2008 02:58:43 GMT

Hi All,After bashing my head against a wall trying to figure out how to recover our indexes with 300 000 transaction and having it take well over 24 hours I have put together some helpful tips to help any poor souls in a similar situation. This was done against 2.1 CE.node.index.FullIndexRecoveryCo

Re: Index Tip Recovery For Large Repositories

andy — Wed, 20 Feb 2008 12:51:38 GMT

Hi

I have rasied an enhancement request to merge many small transactions together during index recovery.

The merge factor is not likely to make much difference for small transactions.

Moving to lucene 2.3 is likely to give significantly better performance.

Your perfromance seems very slow. What hardware are you using?
Where is the alf data stored? What DB? Have you configured to do most content transforms atomically - or only "short" ones (the default). Roughly how many files/folders per transaction?

Andy

Re: Index Tip Recovery For Large Repositories

simon — Fri, 21 Mar 2008 09:14:09 GMT

Hi callermd,

This is an extremely valuable post, thanks! We recently rebuild our indexes (139,071 transactions) and that takes 10 hours on our server so next time we'll try your solution, 10 hours downtime for something like this is not really acceptable. It would be nice if Alfresco could tune this for the following release, hint, hint.

Can you tell us what the performance gain was after your tweaks? How much time did you save after this?

Re: Index Tip Recovery For Large Repositories

andy — Tue, 15 Jul 2008 10:17:52 GMT

Hi

The index rebuild is now multi-threaded.

Andy

Re: Index Tip Recovery For Large Repositories

callermd — Tue, 05 Aug 2008 21:13:23 GMT

Hi

The index rebuild is now multi-threaded.

Andy

What version will have this functionality?

Thanks!

Re: Index Tip Recovery For Large Repositories

pmonks — Tue, 05 Aug 2008 23:43:16 GMT

Putting a backup process in place (as described at http://wiki.alfresco.com/wiki/Backup_and_Restore) can help avoid these problems. See http://wiki.alfresco.com/wiki/Backup_and_Restore#Lucene_Index_Restoration specifically.

Cheers,
Peter

Re: Index Tip Recovery For Large Repositories

oznevo — Sat, 04 Jul 2009 01:47:44 GMT

You should probably add to that:
http://forums.alfresco.com/en/viewtopic.php?f=4&t=15401
We are running on 5-10m transactions…

Oz