FULL Reindex OutOfMemory

[luca]

Hello,
I'm tryng do do a FULL Reindex, but Alfresco run out of memory in an hour.

In the repository there are 163041 documents that occupy 200GB  and 429,598 to be reindex.
I'm using Alfresco 3.2 and, when i do the reindex, the JVM has 8GB of memory.

I set up this parameter in alfresco-global.properties:

index.recovery.mode=FULL
lucene.indexer.batchSize=1000
lucene.indexer.mergeFactor=100
lucene.indexer.minMergeDocs=1000
lucene.indexer.maxMergeDocs=100000‍‍‍‍‍

But the reindexing process runs out of memory at about 70%, this is the log:

15:53:58,400  WARN  [ehcache.distribution.RMISynchronousCacheReplicator] Object with key sysAdminCache.authAllowedUsers is not Serializable and cannot be updated via copy
15:54:00,238  DEBUG [content.cleanup.TrashcanCleaner] Deleted items will be protected during 14 days
15:54:03,131  INFO  [domain.schema.SchemaBootstrap] Schema managed by database dialect org.hibernate.dialect.PostgreSQLDialect.
15:54:09,650  INFO  [domain.schema.SchemaBootstrap] No changes were made to the schema.
15:54:17,201 User:System INFO  [node.index.FullIndexRecoveryComponent] Index recovery started: 429,598 transactions.
15:54:19,993  INFO  [management.subsystems.ChildApplicationContextFactory] Starting 'thirdparty' subsystem, ID: [default]
15:54:20,064  INFO  [alfresco.config.JndiPropertyPlaceholderConfigurer] Loading properties file from class path resource [alfresco/alfresco-shared.properties]
15:54:20,542  INFO  [management.subsystems.ChildApplicationContextFactory] Startup of 'thirdparty' subsystem, ID: [default] complete
### Excluding compile: org.alfresco.repo.search.impl.lucene.index.IndexInfo$Merger::mergeDeletions
15:58:36,955 User:System INFO  [node.index.FullIndexRecoveryComponent]    10 % complete.
16:07:25,902 User:System INFO  [node.index.FullIndexRecoveryComponent]    20 % complete.
16:15:01,312 User:System INFO  [node.index.FullIndexRecoveryComponent]    30 % complete.
16:24:45,491 User:System INFO  [node.index.FullIndexRecoveryComponent]    40 % complete.
16:35:15,831 User:System INFO  [node.index.FullIndexRecoveryComponent]    50 % complete.
16:47:08,152 User:System INFO  [node.index.FullIndexRecoveryComponent]    60 % complete.
### Excluding compile: org.alfresco.repo.search.impl.lucene.index.IndexInfo$Merger::mergeIndexes
17:00:22,170 User:System INFO  [node.index.FullIndexRecoveryComponent]    70 % complete.‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Can anyone help me?

[openpj]

Try to add the following JVM parameters:

-server -Xcomp -Xbatch -Xss1M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
‍‍‍

It should allows you to finish the total reindex, I think that in the latest versions of Alfresco you don't need to add these parameters, but it could depend on your operating system and your JDK version and vendor.

Hope this helps.

[luca]

Thank you very much, I finally and SUCCESFULLY ended the full reindex!

Some numbers:
I started the reindex with 14GB of RAM using the properties suggested by OpenPj like this:
export JAVA_OPTS='-server -Xcomp -Xbatch -Xss1M -Xms14336m -Xmx14336m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:MaxPermSize=128m'

There were 494,584 transactions to reindex and the whole process took 3 hours. The memory occupied remained under 70%.

Next step: upgrade from Alfresco 3.2 to Alfresco 4

In previous attempt to upgrade from Alfresco 3.2 to Alfresco 3.3 or Alfresco 3.4 I noticed that some documents were missing, maybe I did something wrong.
It is possible that this was due to some error in the indexes?