topic Re: Transform core out of memory after migrating from alfresco 4.2 in Alfresco Forum

Transform core out of memory after migrating from alfresco 4.2

fgz01 — Wed, 12 Jul 2023 13:48:05 GMT

Hello,

We are currently in the process of migrating our old 4.2.f alfresco instance to the more recent release in commutiny version.

We have been sucessful migrating to 5.2 then using this great tutorial (https://www.youtube.com/watch?v=kHwq_f9PzYU) we have been able to migrate to Alfresco 7.0 and are currently performing some tests as this version is really different from our previous one.

The problem we are encountering is that the transform-core-aio docker is always crashing with an out of memory reason (visible in /var/log/syslog) with the default configuration in less than a minute.

As our test server have 32Gb of RAM we removed the mem_limit on all dockers to see if it's a temporary issue due to the migration of a big amount of data from our previous server ( ~ 400 Gb).

The fact is that the transform docker is using several Gb, and eventually crash when it's reaching 18 GB RAM in use approximately.

We currently don't really understand why is this occuring, and are convinced that this is not normal as the default configuration (https://github.com/Alfresco/alfresco-docker-installer/blob/master/generators/app/templates/7.0/docker-compose.yml) uses a mem_limit of 2GB.

What is this docker actually used for ? Is this mandatory ?

It seems there is a memory leak somewhere, but sometimes the docker memory seems to be stable between 8 and 9 Gb of RAM.

We tried to modify a bit the docker configuration using this tip : https://hub.alfresco.com/t5/alfresco-content-services-forum/transform-service-unlimited-database-connections/td-p/314523 but it seems to be in the same state.

The transform docker version we are using is 2.3.10 but we also tried using others version of the images (that we found here https://github.com/Alfresco/alfresco-docker-installer/blob/master/generators/app/templates/7.1/docker-compose.yml for example) from more recent alfresco versions but the issue is the same.

Can this be due to the amount of data we are migrating ?

Do you have any idea about a possible cause of the issue ?

Thank you very much

Re: Transform core out of memory after migrating from alfresco 4.2

angelborroy — Thu, 13 Jul 2023 06:27:45 GMT

When upgrading, all the content needs to be reindexed. That means that Transform Service (transform-core-aio component in Docker) is getting many parallel requests during this process. You need to control the load of this parallelism according to your system resources.

Define the number of parallel threads by adding these environment variables to you Transform Service:

-Dserver.tomcat.threads.max=12
-Dserver.tomcat.threads.min=4

Sample configuration is available in https://github.com/Alfresco/alfresco-docker-installer/blob/master/generators/app/templates/7.4/docker-compose.yml#L129C1-L130C44

Re: Transform core out of memory after migrating from alfresco 4.2

fgz01 — Thu, 13 Jul 2023 09:25:53 GMT

Hello @angelborroy

Thank you very much for your answer.

Actually we tried these parameters already and with 2048m for mem_limit the docker is crashing almost every minute.
But as you said this is probably due to the data migration and reindexation. I thought this was only needed for solr.

So we inscreased the mem_limit, applied the parameters you shared along with a restart:always policy and we will let the server live for few days to see if it's stabilize at some point.