Bulk upload in parallel

franciscoduarte — Fri, 04 Mar 2022 11:09:10 GMT

Hello,

We are having some issues when uploading files in bulk from our ADF.

We have two people uploading 10000 files each from our ADF, at the same time. And sometimes, some files fail to upload (usually 2 or 3 files) and we get the following error:

Logs

Can you help?

Thank you

Re: Bulk upload in parallel

abhinavmishra14 — Fri, 04 Mar 2022 23:21:08 GMT

The error indicates that it is trying to modify the same node concurrently or may be a deadlock situation is occurring.

This error also notable : Transactional update cache 'org.alfresco.userToAuthorityTransactionalCache' is full (100).

Increasing the maxItems count can be a quick fix, but does not help, batch processing is the right option.

https://github.com/Alfresco/alfresco-community-repo/blob/master/repository/src/main/resources/alfresco/tx-cache-context.xml#L266

https://github.com/Alfresco/alfresco-community-repo/blob/master/repository/src/main/resources/alfresco/tx-cache-context.xml#L273

cache.userToAuthoritySharedCache.tx.maxItems=100

https://github.com/Alfresco/alfresco-community-repo/blob/master/repository/src/main/resources/alfresco/caches.properties#L214

You may need to upload the files in batches or wrap the the upload logic in transaction scope.

topic Bulk upload in parallel in Alfresco Forum

Bulk upload in parallel

Re: Bulk upload in parallel