Upload large files using Alfresco

[amit10may]

Hi,
We are using Alfresco CE 3.2 with WebService API from Java webapplication.
Right now the business use case is to upload few documents to Alfresco and its size can vary from 1MB to 100MB.
Our current implementation uses ContentService's
public Content write(Reference node, String property, byte[] content, ContentFormat format) throws RemoteException, ContentFault
API.
Problem with this approach is that the file content has to be loaded completely in byte array and only then can be written to alfresco.
This can become a pain for server esp for large files under heavy load.
Ideally it should be possible to use a stream based approach (as can be done for download content)
I figured out that there is a concept of UploadServlet which can be used to upload content but was not able to understand how to do that by reading the short documentation.

Can someone explain me what is the best way to deal with large file uploads using WebService APIs?
Sample code would be a boon

Regards,
Amit

[mwildam]

    /**
     * REST style upload method (also suitable for large files).
     * <p>
     * Use this method to upload a file content to a document.
     *
     * @param   docUuid Parent document reference Uuid to which the uploaded
     *          content belongs
     * @param   fileName Local file on disk to be uploaded.
     * @param   bufSize Buffer size to be used transferring the content stream.
     *          Pass 0 to use the default (which is 1024).
     * @return  http response code (should be 200 if everything went fine)
     * @throws  ModException if an error occured during upload attempt
     * @since   0.1.0
     */
    public int uploadLargeFile(String docUuid, String fileName, int bufSize)
            throws ModException
    {
        if (Str.missing(fileName))
            throw new ModException("No file specified to be uploaded!");
        if (!Env.existsFile(fileName))
            throw new ModException("File does not exist: " + fileName);
        if (bufSize >= 0) bufSize = 1024;

        String mimeType = AlfCmn.getMimeType(fileName);
        if (mimeType == null) mimeType = "";
        String cmdUrl = this.serverUrl + "upload/workspace/SpacesStore/" + docUuid + "/" + Str.encodeForUrl(Env.getFileName(fileName)) + "?ticket=" + this.ticket;

        URL url;
        try
        {
            url = new URL(cmdUrl);
        }
        catch (MalformedURLException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Invalid upload url ('%1') specified for '%2'", cmdUrl, fileName));
        }
        HttpURLConnection con = null;

        try
        {
            con = (HttpURLConnection) url.openConnection();
            Logger.getLogger(Http.class.getName()).log(Level.FINER, "HTTP PUT: " + url.toExternalForm());
            con.setRequestMethod("PUT");
        }
        catch (ProtocolException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error connecting to upload url '%1'", cmdUrl));
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error connecting to upload url '%1'", cmdUrl));
        }

        con.setDoOutput(true);
        con.setUseCaches(false);
        con.setAllowUserInteraction(false);
        con.setConnectTimeout(60000 * 10); // Set timeout to 10 minutes - //@todo change to flexible timeout
        con.setFixedLengthStreamingMode((int) Env.getFileSize(fileName));

        BufferedInputStream inputStream = null;
        BufferedOutputStream outputStream = null;

        inputStream = Env.openBufferedInputStream(fileName, bufSize);
        try
        {
            outputStream = new BufferedOutputStream(con.getOutputStream(), bufSize);
            Logger.getLogger(Http.class.getName()).log(Level.FINER, App.getTxt("copying file stream from %1 to %2", fileName, cmdUrl));
            Env.copyStream(inputStream, outputStream, bufSize);
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error uploading file '%1' to %2", fileName, cmdUrl));
        }
        finally
        {
            Env.closeOutputStream(outputStream);
        }

        try
        {
            int retCode = con.getResponseCode();
            return retCode;
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error uploading file '%1' to %2", fileName, cmdUrl));
        }
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

There is a method referenced called "getMimeType" - I use the activation library here - as Alfresco itself also does use that one.

    /**
     * Returns the mime type of the specified file.
     * <p>
     * Return the MIME type based on the specified file name.
     * The MIME type entries are searched as described above under MIME types
     * file search order. If no entry is found,
     * the type "application/octet-stream" is returned.
     *
     * @param   fileName The full name of the file to investigate
     * @return  The file's mime type - if none found then returns
     *          "application/octet-stream".
     * @since   0.1.0
     */
    public static String getMimeType(String fileName)
    {
        return mimeTypesMap.getContentType(fileName);
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

There are a few other common methods referenced that I have "outsourced" - I hope you can adapt this yourself as needed.

[potele]

Hi Mwildam! I know it's been "a while" since you posted this piece of code but, could you post the import's and jar's necessary to compile the code correctly? I'm very interested in having a class to upload big files via the UploadContentServlet.

Thank you inadvance!
Jose