topic Re: Full Reindex stuck on 30% in Alfresco Archive

Full Reindex stuck on 30%

leonk — Mon, 28 Mar 2011 01:10:23 GMT

Hello,I'm trying to rebuild indexes in 3.4.d version (upgraded from 3.3.g).1) So I've set:index.recovery.mode=FULLin alfresco-global.properties2)deleted contents of dir.indexes3) Increased org.alfresco.cache.node.aspectsTransactionalCache value (cause I was getting messages that it's full during reb

Re: Full Reindex stuck on 30%

loftux — Mon, 28 Mar 2011 05:43:55 GMT

Did it make it to 100% when the transactional cache had its original value? If so, change it back.
What was the transactional message? If it was a WARN, then it is just that, and can be ignored.

And as an advice, rename the old index directory rather than delete. Alfresco will create a new index directory. That way you can restore your old index should anything go wrong. Once you have a new working index, thats the time to delete the old index.

Re: Full Reindex stuck on 30%

leonk — Mon, 28 Mar 2011 10:20:25 GMT

Did it make it to 100% when the transactional cache had its original value? If so, change it back.
What was the transactional message? If it was a WARN, then it is just that, and can be ignored.

And as an advice, rename the old index directory rather than delete. Alfresco will create a new index directory. That way you can restore your old index should anything go wrong. Once you have a new working index, thats the time to delete the old index.

Hi,

It never made it to 100% in both cases
And yes I did see that it's a warning but changing org.alfresco.cache.node.aspectsTransactionalCache didn't help.

I have this index directory backed up, so getting back is not a problem.

The problem is that it gets stuck ;(

Tomorrow I'll try to rename index dir. and report here.

Re: Full Reindex stuck on 30%

andy — Mon, 28 Mar 2011 10:47:22 GMT

Hi

The version you are using does not include a bug fix for reindexing transactions that contain very large cascade deletes.
The delete from the UI will cascade delete optimally but the index rebuild will not.

Is this likely to be your issue?

Andy

Re: Full Reindex stuck on 30%

dranakan — Mon, 28 Mar 2011 14:16:17 GMT

Hello Andy,

The version you are using does not include a bug fix for reindexing transactions that contain very large cascade deletes.

We can not rebuild (and restore) an Alfresco 3.4d if some files have been deleted ?

Re: Full Reindex stuck on 30%

loftux — Mon, 28 Mar 2011 14:52:04 GMT

I found this entry
http://issues.alfresco.com/jira/browse/ALF-5166
Maybe thats the one Andy is referring to.

I to would like to know the answer to this, because it is a really serious matter if you can in fact break a repository completely if you hit this bug.

Can you work around it by emptying/purging all deleted items before doing reindex?

Re: Full Reindex stuck on 30%

leonk — Mon, 28 Mar 2011 22:51:57 GMT

I reckon Andy and Loftux are right.
The thing is that previously our Alfresco was configured to syncronize users from our AD (~180,000 users) and it's created 180,000 User Homes. Apparently Alfresco has some issues with such large number of User Homes.

So I've reconfigured Alfresco to Authenticate (without Sync.) users only and deleted all empty directories with the following script (courtesy of John Jaquette):


var childList = companyhome.childByNamePath("User Homes").children;
for (var i=0; i<childList.length; i++)
{
   var child = childList[i];
   logger.log(child.name);
   var subchildren = child.children;
  logger.log(subchildren.length);
  if (subchildren.length==0){
     child.remove();
     }
}
‍‍‍‍‍‍‍‍‍‍‍‍‍

If this JIRA is right - http://issues.alfresco.com/jira/browse/ALF-5166

Why won't they attach a fixed .jar or .xml file?

P.S.:
Andy may be you can supply us with fixed components as this issue is fixed for Enterprise ver. (I presume it was tested first on Community nightly build)

Re: Full Reindex stuck on 30%

andy — Tue, 29 Mar 2011 10:05:47 GMT

Hi

The issue only affects very large transactions and particularly those with cascade deletes that are optimised at delete time but not when the index is rebuilt.

There were several parts to the fix.

The issue is fixed in the Enterprise product. It will be in the community release after 3.4.
The easiest fix is via support.

If you are doing a full index rebuild you could delete "delete only transactions" to try and work around the problem.

Better to configure the repo not to check the index and set

index.tracking.minRecordPurgeAgeDays=0‍

and wait for the clean up to run.
It will take out deleted nodes and and empty transactions.
Then set tthe configuration back rebuild your index.

See node-services-context.xml

Andy

Re: Full Reindex stuck on 30%

leonk — Wed, 30 Mar 2011 01:50:15 GMT

Hello Andy,

I've added
index.recovery.mode=NONE
index.tracking.minRecordPurgeAgeDays=0

to alfresco-global.properties (Is that what you meant?) files and restarted Alfresco.

Can you tell me when this "cleanup" will run?
Is there a way to trigger this cleanup manually?
How do I know that it's finished and I can set configs back and rebuild the index?

Update:
On the restart I'm getting lots of those warnings:
Alfresco interface gives me the following message:


WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
WARN : org.alfresco.wcm.client.util.impl.GuestSessionFactoryImpl - WQS unable to connect to repository: Not Found
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I'm unable to connect via Share interface.
Alfresco interface gives me the following message:


HTTP Status 404 -

type Status report

message

description The requested resource () is not available.
Apache Tomcat/6.0.26
‍‍‍‍‍‍‍‍‍‍

Thanks,
Leon