Problems with uploading big files in Alfresco

[nettania]

Hello,

we are using alfresco in an architects office and there is a problem when we want to upload files bigger than 4GB which often happens when we get the results of our measuring team, a single file can easily have more than 14GB. The file is completley uploaded and stored in the tomcat/temp folder but at 100% alfresco waits a little bit until it starts again from 0%, this goes on until I stop the file upload manually. Each approach a new file is created in the tomcat/temp folder, and this file is also complete. Therefore the problem cannot be the proxy or tomcat but the repository. Can someone help me, what I should check, the alfresco or share logs does not show any issue related errors? I cannot imagine that we are the only architects who are using alfesco.

Thanks,

Florian

[angelborroy]

Likely this presentation from our colleague @jpotts may help you to explore alternative ways:

https://www.slideshare.net/slideshow/moving-gigantic-files-into-and-out-of-the-alfresco-repository/8...

[nettania]

Hello,

thank you, I already read the slides, but however in my case I have a limit of filesize, about 30GB. And the upload process uisng share works, the file is uploaded into the temp folder of tomcat an than sucessfully moved to the repository. The problem ist the communication between share and the alfresco backend api. The share application shows 100%, waits for some minutes and than restarts the upload process. When I cancel it the share folder is reloaded and it shows the uploaded file in the first attemp. There must be a timeout problem between share and alfresco api, the share error also indicates this:

2024-09-19 14:26:11,083  INFO  [webscripts.connector.RemoteClient] [http-nio-8080-exec-7] Exception calling (POST) http://localhost:8080/alfresco/s/api/upload?Alfresco-CSRFToken=B0MAUSxSV8hhkXEZjLff%2F7VrxI3GhY0%2Fyn7VhGFfPoU%3D&alf_ticket=TICKET_734e65fe4f2f22c25da6d62ff1992a3538dd7a01
 java.net.SocketTimeoutException: Read timed out
        at java.base/java.net.SocketInputStream.socketRead0(Native Method)
        at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

Can someone tell me some settings to prevent this timeout?

Thanks,

Florian

[afaust]

The timeout between Share and Alfresco Repository only has an internal default of 120 seconds, but can be configured by overriding the connector.remoteclient bean prototype (https://github.com/Alfresco/alfresco-community-share/blob/master/share/src/main/resources/alfresco/s...) and setting a readTimeout property. The reason it fails at 100% is that the final byte chunk transfer request has to wait for the Repository to copy the file from the temp folder to the content store, which can take a long time depending on disk performance, and exceed the timeout for that reason.