topic Re: Upload large files using Alfresco in Alfresco Archive

Upload large files using Alfresco

amit10may — Wed, 20 Jan 2010 08:57:10 GMT

Hi,We are using Alfresco CE 3.2 with WebService API from Java webapplication.Right now the business use case is to upload few documents to Alfresco and its size can vary from 1MB to 100MB.Our current implementation uses ContentService'spublic Content write(Reference node, String property, byte[] con

Re: Upload large files using Alfresco

mrogers — Wed, 20 Jan 2010 09:04:01 GMT

WebServices are not suitable for large files. You need to use one of the streaming interfaces.

Re: Upload large files using Alfresco

amit10may — Wed, 20 Jan 2010 09:13:04 GMT

Wow..I did not expect such a quick reply.. so thanks a lot
Can you please give me some pointer to where I can look for help on streaming interface?
I want to integrate it with my Java web application (in fact spring based services are client of the alfresco api)

Re: Upload large files using Alfresco

junieboy — Sat, 23 Jan 2010 08:31:49 GMT

Hi Amit,

Could I ask you in which javadoc would I find the org.alfresco.webservices classes? I am currently using Alfresco CE 3.0 and am not able to see any javadoc for the NamedValue, CMLCreate, UpdateResult, etc? Also, could you further elaborate on the approach below?

Thanks in advance.

Re: Upload large files using Alfresco

junieboy — Sun, 04 Apr 2010 13:13:12 GMT

WebServices are not suitable for large files. You need to use one of the streaming interfaces.

What do you mean when you say "the streaming interfaces" ? Are these the Java InputStream, OutputStream interfaces? Please elaborate. Thanks.

Re: Upload large files using Alfresco

openpj — Sun, 04 Apr 2010 18:56:12 GMT

Yes, you can use the upload servlet to stream the content using a PUT HTTP method:
http://wiki.alfresco.com/wiki/URL_Addressability#UploadContentServlet
Hope this helps.

Re: Upload large files using Alfresco

junieboy — Mon, 05 Apr 2010 05:31:02 GMT

Yes, you can use the upload servlet to stream the content using a PUT HTTP method:
http://wiki.alfresco.com/wiki/URL_Addressability#UploadContentServlet
Hope this helps.

Hi,

How does this impact the attaching of the metadata to the files? We need these functionalities: save big files to Alfresco and use metadata for searching in the content repository (a custom content model will be created for this). The metadata will come from an external system w/c will invoke the Alfresco web services API.

Thanks so much for your response.

Best regards,
Junie

Re: Upload large files using Alfresco

mwildam — Tue, 06 Apr 2010 10:24:28 GMT

How does this impact the attaching of the metadata to the files? We need these functionalities: save big files to Alfresco and use metadata for searching in the content repository (a custom content model will be created for this). The metadata will come from an external system w/c will invoke the Alfresco web services API.

You can combine these APIs - I do also. For the upload via you just specify the documents uuid and upload the file - so it is "compatible" with the rest done through web services API.

Re: Upload large files using Alfresco

junieboy — Wed, 07 Apr 2010 14:05:50 GMT

How does this impact the attaching of the metadata to the files? We need these functionalities: save big files to Alfresco and use metadata for searching in the content repository (a custom content model will be created for this). The metadata will come from an external system w/c will invoke the Alfresco web services API.
You can combine these APIs - I do also. For the upload via you just specify the documents uuid and upload the file - so it is "compatible" with the rest done through web services API.

Hi Martin! Thanks so much for the response. Would you mind if I ask you to post sample code/implementations you have? If it's fine to ask for your source, my email is junieboy@gmail.com.

Regards.

Re: Upload large files using Alfresco

mwildam — Wed, 07 Apr 2010 14:14:08 GMT

    /**
     * REST style upload method (also suitable for large files).
     * <p>
     * Use this method to upload a file content to a document.
     *
     * @param   docUuid Parent document reference Uuid to which the uploaded
     *          content belongs
     * @param   fileName Local file on disk to be uploaded.
     * @param   bufSize Buffer size to be used transferring the content stream.
     *          Pass 0 to use the default (which is 1024).
     * @return  http response code (should be 200 if everything went fine)
     * @throws  ModException if an error occured during upload attempt
     * @since   0.1.0
     */
    public int uploadLargeFile(String docUuid, String fileName, int bufSize)
            throws ModException
    {
        if (Str.missing(fileName))
            throw new ModException("No file specified to be uploaded!");
        if (!Env.existsFile(fileName))
            throw new ModException("File does not exist: " + fileName);
        if (bufSize >= 0) bufSize = 1024;

        String mimeType = AlfCmn.getMimeType(fileName);
        if (mimeType == null) mimeType = "";
        String cmdUrl = this.serverUrl + "upload/workspace/SpacesStore/" + docUuid + "/" + Str.encodeForUrl(Env.getFileName(fileName)) + "?ticket=" + this.ticket;

        URL url;
        try
        {
            url = new URL(cmdUrl);
        }
        catch (MalformedURLException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Invalid upload url ('%1') specified for '%2'", cmdUrl, fileName));
        }
        HttpURLConnection con = null;

        try
        {
            con = (HttpURLConnection) url.openConnection();
            Logger.getLogger(Http.class.getName()).log(Level.FINER, "HTTP PUT: " + url.toExternalForm());
            con.setRequestMethod("PUT");
        }
        catch (ProtocolException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error connecting to upload url '%1'", cmdUrl));
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error connecting to upload url '%1'", cmdUrl));
        }

        con.setDoOutput(true);
        con.setUseCaches(false);
        con.setAllowUserInteraction(false);
        con.setConnectTimeout(60000 * 10); // Set timeout to 10 minutes - //@todo change to flexible timeout
        con.setFixedLengthStreamingMode((int) Env.getFileSize(fileName));

        BufferedInputStream inputStream = null;
        BufferedOutputStream outputStream = null;

        inputStream = Env.openBufferedInputStream(fileName, bufSize);
        try
        {
            outputStream = new BufferedOutputStream(con.getOutputStream(), bufSize);
            Logger.getLogger(Http.class.getName()).log(Level.FINER, App.getTxt("copying file stream from %1 to %2", fileName, cmdUrl));
            Env.copyStream(inputStream, outputStream, bufSize);
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error uploading file '%1' to %2", fileName, cmdUrl));
        }
        finally
        {
            Env.closeOutputStream(outputStream);
        }

        try
        {
            int retCode = con.getResponseCode();
            return retCode;
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error uploading file '%1' to %2", fileName, cmdUrl));
        }
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

There is a method referenced called "getMimeType" - I use the activation library here - as Alfresco itself also does use that one.

    /**
     * Returns the mime type of the specified file.
     * <p>
     * Return the MIME type based on the specified file name.
     * The MIME type entries are searched as described above under MIME types
     * file search order. If no entry is found,
     * the type "application/octet-stream" is returned.
     *
     * @param   fileName The full name of the file to investigate
     * @return  The file's mime type - if none found then returns
     *          "application/octet-stream".
     * @since   0.1.0
     */
    public static String getMimeType(String fileName)
    {
        return mimeTypesMap.getContentType(fileName);
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

There are a few other common methods referenced that I have "outsourced" - I hope you can adapt this yourself as needed.

Re: Upload large files using Alfresco

potele — Mon, 18 Feb 2013 11:05:09 GMT

Hi Mwildam! I know it's been "a while" since you posted this piece of code but, could you post the import's and jar's necessary to compile the code correctly? I'm very interested in having a class to upload big files via the UploadContentServlet.

Thank you inadvance!
Jose

Re: Upload large files using Alfresco

akoudrin — Thu, 14 Nov 2013 02:52:04 GMT

Could you tell me which interface is best to use to upload a 200MB file to Alfresco4.0.c? If you have a code snippet (including the imports) that would be even better.