topic Alfresco write-read 500 MO when adding a 200 ko pdf document in Alfresco Archive

Alfresco write-read 500 MO when adding a 200 ko pdf document

dbas2000 — Tue, 04 Sep 2012 10:16:50 GMT

Our Alfresco output 500 MO to and from the disk where lucene-indexes are located, for each document added to the repository.Our documents are only small pieces of PDFA files (200 ko) !So, it takes more than 10 seconds to process a document and 2000 documents are inserted per days.This scenario appea

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

mrogers — Tue, 04 Sep 2012 16:18:11 GMT

The indexes are used for more than just full text search.

And although Lucene is IO heavy it should not be a problem.

The recommendation I've always seen is to put your indexes on your fastest local disk.
And make sure its not configured with a filesystem with un-neccessasary features like slow and safe RAID settings or virus scanner. The index is derived data, it can be regenerated.

The usual problems with running alfresco in a JVM is too little RAM.

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

t_sato — Wed, 05 Sep 2012 08:51:57 GMT

Hi,

I looking for a solution to reduce these IO writing, any help welcome !

How did you know the absolute number of write I/O is your problem?

I guess your problem is indexing speed, correct? Then, sync is likely the most time consuming operations as index file gets fragmented. And it would be mostly caused by index merger, which could be improved by index writer's buffer RAM size. Could be only improved by some kind of sharding if you don't change your hardware.

But anyway, I think you need to make your problem clear and nail down further to find your real bottleneck.

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

dbas2000 — Mon, 10 Sep 2012 13:09:41 GMT

Thanks to mrogers and t_sato for your post.

After more investigations, I discovered that is the merge process that produce this traffic.

With the tool iotop, we can see that the IO traffic is produced whatever segments are merged.

The mergeFactor is fixed to 5 but every 2 documents a segment is flushed to the disk.

Below is the result from the IndexInfo.jar against the SpacesStore.

Does somebody knows how to analyse these output ?

The contentStore holds 1'600'000 documents and the first segment holds 3'029'531 documents !

java -Xmx512m -cp ./IndexInfo.jar org.alfresco.repo.search.impl.lucene.index.IndexInfo C:\\labs\\backup-lucene-indexes\\workspace\\SpacesStore  
Entry List for lucene-indexes\workspace\SpacesStore ‍‍

   Size = 9
        0        Name=0ab76c3f-be42-4486-bf5d-0716df6084f9 Type=INDEX Status=COMMITTED Docs=3029531 Deletions=0
        1        Name=12f3f5b3-047b-4179-808a-c5e55dab6ba4 Type=INDEX Status=COMMITTED Docs=1745366 Deletions=0
        2        Name=adfdcb27-3e94-4cfe-9826-072335af142b Type=INDEX Status=COMMITTED Docs=1128625 Deletions=0
        3        Name=3ad8cb8c-3515-47e2-b26b-6c342e8a5b26 Type=INDEX Status=COMMITTED Docs=725190 Deletions=0
        4        Name=c15f8d81-79ed-4a55-985a-1f7b8a1a1188 Type=INDEX Status=COMMITTED Docs=316397 Deletions=0
        5        Name=9d21316f-64b4-406c-8029-011bbc029970 Type=DELTA Status=COMMITTED Docs=40 Deletions=29
        6        Name=de201603-4374-4cf1-975c-673fe5cc2d7b Type=DELTA Status=COMMITTED Docs=80 Deletions=65
        7        Name=d85fb371-5b9f-4938-a3e2-4d7aac73c5ff Type=DELTA Status=COMMITTED Docs=128 Deletions=86
        8        Name=5b1d1df9-3e96-4334-9914-0fd2e996a775 Type=DELTA Status=COMMITTED Docs=119 Deletions=119‍‍‍‍‍‍‍‍‍‍

Thank's for any help.

Pierre-André

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

t_sato — Tue, 11 Sep 2012 04:10:36 GMT

Hi Pierre,

So now this must be the wiki you want to take a look.

It says:

It is usualy a good practice that the highest-numbered INDEX entries (which contains the least documents, number 4 in the examples above) do not contain more than a few hundred documents. If it's not the case, it could lead to massive amount of IO pressure on the index directories for merging operations.

You have the default merge factor, 5, so it applies to you as well. You have 316397 documents in the highest-numbered INDEX entry. It is far more than hundreds! So increasing the merge factor is definitely an option to improve indexing performance.

There Lucene pages are also helpful to understand how Lucene index works.
IndexWriter(2.9.3)
Improve Index Speed from Lucene wiki
Full Text Search Configuraiton from Alfresco wiki

Hope this helps.

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

dbas2000 — Tue, 11 Sep 2012 06:19:39 GMT

Hi t_sato,

So I changed the configuration with this parameters and the FULL reindexing process is currently working - Wait and see !

lucene.indexer.mergerTargetIndexCount=8
lucene.indexer.maxRamInMbForInMemoryMerge=48
lucene.indexer.writerRamBufferSizeMb=48
lucene.indexer.mergerRamBufferSizeMb=48
lucene.indexer.maxRamInMbForInMemoryIndex=48‍‍‍‍‍

I will post the result re-indexing is completed.

Pierre-André

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

dbas2000 — Thu, 13 Sep 2012 13:06:46 GMT

Hi,

Here is the indexes status after the FULL indexes rebuilding. 24 hours for 1'600'000 documents :

Size = 7
        0        Name=facae57d-2997-4964-b710-6bf7a88e1345 Type=INDEX Status=COMMITTED Docs=5527866 Deletions=0
        1        Name=32064b10-d4dd-4514-a652-5bd55921892a Type=INDEX Status=COMMITTED Docs=845705 Deletions=0
        2        Name=a68a0558-9429-4217-a44e-7f696e79a113 Type=INDEX Status=COMMITTED Docs=517370 Deletions=0
        3        Name=0482c5b5-0c3f-4c2d-ae87-ad848466faac Type=INDEX Status=COMMITTED Docs=49291 Deletions=0
        4        Name=d0b834a1-ce86-4493-bd13-a1905360c893 Type=INDEX Status=COMMITTED Docs=4751 Deletions=0
        5        Name=27e26039-1ce6-4336-bea3-dd45f9fc4533 Type=DELTA Status=COMMITTED Docs=18 Deletions=11
        6        Name=0563c9a4-daf6-4dea-91a1-96d1936cf651 Type=DELTA Status=COMMITTED Docs=50 Deletions=50‍‍‍‍‍‍‍‍

I thing it's a lot better than the previous status !

My questions are :

In order to keep the production online, does it possible to launch the reindexing process in an other server and then replace de production indexes with the rebuilded indexes ?

In the first segment there is Docs=5'527'866. Does Docs represents the number of indexed files, because the contenstore hold only 1'600'000 documents ?

Thanks

Pierre-André

Re: Alfresco write-read 500 MO when adding a 200 ko pdf document

t_sato — Fri, 14 Sep 2012 03:09:25 GMT

Looks good!

In order to keep the production online, does it possible to launch the reindexing process in an other server and then replace de production indexes with the rebuilded indexes ?

I'm afraid not unless you can have your system read-only during the rebuilding process(24 hours?).

You may need a professional Alfresco consulting service for the best possible deployment in production.

In the first segment there is Docs=5'527'866. Does Docs represents the number of indexed files, because the contenstore hold only 1'600'000 documents ?

I guess DOCS comes from numDocs of IndexReader. Then yes, it is indexed documents. I don't know details, but it means(not surprising) that 2 or 3 additional system document are stored per user document.