out of heap space when uploading large files

alanbell — Fri, 24 Dec 2010 10:36:48 GMT

I am using the python-cmis library to upload files to Alfresco community 3.4.b running on Ubuntu server.where /mnt/import/bigfile is a 298MB file. The file can be imported successfully using the share interface or classic Alfresco web UI and small files of a few MB in size are uploaded by this code

Re: out of heap space when uploading large files

christophe — Tue, 30 Aug 2011 09:32:25 GMT

Hello

I am having a similar issue while upload a 100Mb file with Apache Chemistry CMIS library.
I am using the AtomPub protocol.

Here is the stack trace client side :

Exception in thread "main" org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeException: Erreur Interne de Servlet
   at org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.convertStatusCode(AbstractAtomPubService.java:450)
   at org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.post(AbstractAtomPubService.java:568)
   at org.apache.chemistry.opencmis.client.bindings.spi.atompub.ObjectServiceImpl.createDocument(ObjectServiceImpl.java:109)
   at org.apache.chemistry.opencmis.client.runtime.SessionImpl.createDocument(SessionImpl.java:650)
   at org.apache.chemistry.opencmis.client.runtime.FolderImpl.createDocument(FolderImpl.java:87)
   at org.apache.chemistry.opencmis.client.runtime.FolderImpl.createDocument(FolderImpl.java:409)
   at com.mycompany.alfresco.cmis.main.create.CreatePolar.doit(CreatePolar.java:54)
   at com.mycompany.alfresco.cmis.main.AbstractAction.run(AbstractAction.java:12)
   at java.lang.Thread.run(Thread.java:662)
   at com.mycompany.alfresco.cmis.main.SimpleMain.main(SimpleMain.java:24)‍‍‍‍‍‍‍‍‍‍‍

server side :

11:20:57,216  ERROR [extensions.webscripts.AbstractRuntime] Exception from executeScript - redirecting to status template error: 07300001 Wrapped Exception (with status template): 
07302060 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js': 
07302059 Java heap space org.springframework.extensions.webscripts.WebScriptException: 
07300001 Wrapped Exception (with status template): 07302060 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js': 
07302059 Java heap space
        at org.springframework.extensions.webscripts.AbstractWebScript.createStatusException(AbstractWebScript.java:758)
        at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:171)
        at org.alfresco.repo.web.scripts.RepositoryContainer$2.execute(RepositoryContainer.java:383)
        at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:381)
        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:436)
        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:466)
        at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:304)
        at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:352)
        at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:189)
        at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

the root cause is :

Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3209)
        at java.lang.String.<init>(String.java:215)
        at java.lang.StringBuffer.toString(StringBuffer.java:585)
        at com.ctc.wstx.util.TextBuffer.contentsAsString(TextBuffer.java:403)
        at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:821)
        at org.apache.axiom.om.impl.builder.StAXBuilder.createOMText(StAXBuilder.java:245)
        at org.apache.axiom.om.impl.builder.StAXBuilder.createOMText(StAXBuilder.java:216)
        at org.apache.abdera.parser.stax.FOMBuilder.applyTextFilter(FOMBuilder.java:158)
        at org.apache.abdera.parser.stax.FOMBuilder.next(FOMBuilder.java:206)
        at org.apache.axiom.om.impl.llom.OMElementImpl.getNextOMSibling(OMElementImpl.java:265)
        at org.apache.axiom.om.impl.traverse.OMChildrenQNameIterator.hasNext(OMChildrenQNameIterator.java:69)
        at org.apache.axiom.om.impl.llom.OMElementImpl.getFirstChildWithName(OMElementImpl.java:223)
        at org.apache.abdera.parser.stax.FOMExtensibleElement.getExtension(FOMExtensibleElement.java:91)
        at sun.reflect.GeneratedMethodAccessor979.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
        at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:243)
        at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

It looks like Alfresco tries to put the whole content in memory and it leads to a OutOfMemoryError.

The server is running with +/- 980Mb of memory

When I am using the SOAP protocol, everything just works file (I am able to upload really large files (1Gb+).

I would like to use the AtomPub protocol as it is less verbose, but this issue is blocking as the application will be uploading large files.

Any though on that ?

Re: out of heap space when uploading large files

victorba — Fri, 30 Aug 2013 11:57:16 GMT

Hello Christophe,

did you find a solution to upload large files using Atompub protocol? I have the same problem using Alfresco Community 4.2

topic out of heap space when uploading large files in Alfresco Archive

out of heap space when uploading large files

Re: out of heap space when uploading large files

Re: out of heap space when uploading large files