topic Re: Indexing questions... in Alfresco Archive

Indexing questions...

jonnycat26 — Tue, 05 Feb 2008 14:48:27 GMT

Hi,We've been using Alfresco as our school CMS system, and we experience a problem yesterday that I suspect is related to indexing, and I was hoping for some input.We had a user upload a bit over 800MB of files via SMB, and immediately after the CPU spiked and the machine became unresponsive. I sus

Re: Indexing questions...

ebo — Wed, 06 Feb 2008 19:16:57 GMT

Not an expert myself but this may be useful

C:\Alfresco\tomcat\webapps\alfresco\WEB-INF\classes\alfresco\repository.properties

This file has a bunch of lucene index properties in it. Also, check out

http://wiki.alfresco.com/wiki/Data_Dictionary_Guide

and search for the word index … you can set the atomic to be false so lucene indexes run in the background.

Hope this helps and puts you on the right path.

Re: Indexing questions...

andy — Tue, 19 Feb 2008 14:43:08 GMT

Hi

CIFS indexes as it goes (it does not work as one big transaction and index at the end). Content will most likely be indexed in the background.
This will happen as it goes (after up to a minute)

So it is likely to be something like the conversion of content to text in the background, running rules etc. Background indexing is fine unless you have some big conversion that eats all your available memory.

Are you using OpenOffice for conversions?
What kind of docs were loaded?
Can you report the memory, CPU and IO load?
Have you got any rules/actions set up?
Can you get a snapshot of what is going on using jstack?

Andy

Re: Indexing questions...

jonnycat26 — Tue, 19 Feb 2008 14:48:18 GMT

Are you using OpenOffice for conversions?
What kind of docs were loaded?
Can you report the memory, CPU and IO load?
Have you got any rules/actions set up?
Can you get a snapshot of what is going on using jstack?

Andy,

Yes, we are using OpenOffice for conversions, and these were all Word docs. We had about 800MB uploaded in roughly 12 mins. I can't report specifically on the CPU and IO load because we're running Alfresco in an ESX cluster and I find the information to be less than totally reliable.

We do have 768MB of RAM on the box, and we're running Alfresco with it's default out-of-the-box config. I'm starting to think we need to allocate more memory to the box and JVM, but that's just a guess and any input is appreciated.

-jon

Re: Indexing questions...

andy — Wed, 27 Feb 2008 10:20:43 GMT

Hi

Is the database on the same machine/vmware image?

I think you need more memory and if possible a few more CPUs. Especially if the DB is on the same machine. If you only have 768M on the machine or VMWare image you will likely end up swapping in VMWare.

VMWare IO access will be slow compared with native IO. The DB and lucene indexing will be affected by this. If you go over your memory size and end up swapping then my guess is everything will stop.

Andy

Re: Indexing questions...

jonnycat26 — Mon, 17 Mar 2008 15:31:03 GMT

Hi

Is the database on the same machine/vmware image?

I think you need more memory and if possible a few more CPUs. Especially if the DB is on the same machine. If you only have 768M on the machine or VMWare image you will likely end up swapping in VMWare.

Andy,

No, the database is on another machine, and we just reconfigured the machine and upped the RAM to 2GB and gave the JVM 1.2gb of memory. FYI, it's configured as a 2CPU machine, and the rest of the cluster is pretty idle, so this machine isn't being starved for CPU.

We're still having issues tho, when uploading large amounts of files via CIFS. For each file we upload, the CPU does spike, and after a while, the entire JVM comes down. Is there any way I can tune the indexing (I have set atomic to false) to make it much, much, much less aggressive?

Re: Indexing questions...

bejelith — Tue, 23 Sep 2008 09:44:02 GMT

No, the database is on another machine, and we just reconfigured the machine and upped the RAM to 2GB and gave the JVM 1.2gb of memory. FYI, it's configured as a 2CPU machine, and the rest of the cluster is pretty idle, so this machine isn't being starved for CPU.

We're still having issues tho, when uploading large amounts of files via CIFS. For each file we upload, the CPU does spike, and after a while, the entire JVM comes down. Is there any way I can tune the indexing (I have set atomic to false) to make it much, much, much less aggressive?

Same problem here!!