topic Re: Strange and serious re-indexing behaviour in Alfresco Archive

Strange and serious re-indexing behaviour

atariq — Thu, 31 Jan 2008 16:05:20 GMT

We've recently and strangely been having issues re-indexing our repository and it's had severe problems for the team.Our repository is stored on another fileserver while the lucene-indexes were stored on the local Alfresco server. In an attempt to rebuild the index, I stopped Alfresco, removed the c

Re: Strange and serious re-indexing behaviour

atariq — Thu, 31 Jan 2008 20:11:47 GMT

Ok, the situation is under control now. I opted to restore an older index of the repository and it thankfully worked. Albeit with the loss of a few files that I was able to retrieve via the Manage Deleted Items feature in the web-client. As long as the department is able to access the documents everyone is happy!

Although, that was a bit of a scare though. I'm still not sure why the lucene-indexes folder was pretty much empty after apparently "successfully" rebuilding the index.

In addition, I'm seeing a strange discrepancy between CIFS and the web client. All the up-to-date files can be viewed and downloaded via the web client, however, if I go through CIFS, the directories all seem to be at least a couple of months old. Any ideas why this would be happening?

Re: Strange and serious re-indexing behaviour

rdanner — Fri, 01 Feb 2008 03:10:20 GMT

Ok, the situation is under control now. I opted to restore an older index of the repository and it thankfully worked. Albeit with the loss of a few files that I was able to retrieve via the Manage Deleted Items feature in the web-client. As long as the department is able to access the documents everyone is happy!

Although, that was a bit of a scare though. I'm still not sure why the lucene-indexes folder was pretty much empty after apparently "successfully" rebuilding the index.

In addition, I'm seeing a strange discrepancy between CIFS and the web client. All the up-to-date files can be viewed and downloaded via the web client, however, if I go through CIFS, the directories all seem to be at least a couple of months old. Any ideas why this would be happening?

I'm glad you where able to recover – it's this kinda stuff that having a support contract makes me feel a bit better. I'd like to know more about what happened and what really went wrong here. The CIFS projection issue is a bit troubling. Please keep us up to date as you discover anything. I hope that anyone who has seen similar behavior will chime in here.

Re: Strange and serious re-indexing behaviour

atariq — Mon, 04 Feb 2008 03:19:14 GMT

Yup, Russ, it's quite alarming. I've done tests again to tackle the issues I saw, and they're being replicated without a solution (other than using an older backup of the lucene indexes). So I guess my questions are two-fold:

1) Why aren't the indexes being rebuilt fully and properly after I set the index.recovery.mode flag to FULL? I see no errors in the logs, the process ends by saying:

14:53:37,704 INFO [node.index.FullIndexRecoveryComponent]      20 % complete.
14:56:30,202 INFO [node.index.FullIndexRecoveryComponent]      30 % complete.
14:59:22,320 INFO [node.index.FullIndexRecoveryComponent]      40 % complete.
15:11:03,774 INFO [node.index.FullIndexRecoveryComponent]      50 % complete.
15:40:00,134 INFO [node.index.FullIndexRecoveryComponent]      60 % complete.
16:23:45,650 INFO [node.index.FullIndexRecoveryComponent]      70 % complete.
17:24:10,075 INFO [node.index.FullIndexRecoveryComponent]      80 % complete.
18:41:00,758 INFO [node.index.FullIndexRecoveryComponent]      90 % complete.
20:04:15,408 INFO [node.index.FullIndexRecoveryComponent]      100 % complete.
20:04:15,428 INFO [node.index.FullIndexRecoveryComponent] Index recovery completed.

yet the lucene-indexes folder that gets created only has the directory tree structure and lacks any files. Why would this be happening?

and

2) Why would going through CIFS in Windows Explorer display files dated months ago while the web client is showing all the up-to-date data items? As such, CIFS is rendered useless for our users and in fact poses a danger to our data integrity.

I'm just trying to make sense of the situation.

Re: Strange and serious re-indexing behaviour

andy — Tue, 12 Feb 2008 16:46:07 GMT

Hi

The index rebuild may not have indexed all the content. If the transformation to text is expected to take more than 20ms (by default) then this is done in the background - and may happen after the rebuild says it is done. When it is done - all the atomically indexed properties are done - the file content may be done in the background.

It is clearly faster to do an index catch up from a backup as there is less to do.

The default property that controls this is in repository.properties. You can increase this and effectively make all content indexed atomically.

The content index is a big chunk of the index size.

There should be files in the index structiure.

Andy