I am looking for an approach to upload large number of files from the users local machine to a remote Nuxeo instance/server deployed in AWS cloud. The files are residing in multiple folders which are nested as well. Total size to upload are in gigs. Assuming we will get a better network speed from the local machine, what will be best approach to implement this.
Approach we tried and help required are :
Nuxeo Bulk Importer : This addon requires the files to be in a path accessible to the Nuxeo server. Though this supports uploading of files and folders, but will be difficult to copy all the files from local machine of multiple users to a drive of server which hosts Nuxeo instance and make Nuxeo to read it automatically through from Jobs . So we planned to upload all files to S3 bucket (using S3 plugin in webui) and try to make bulk importer read from there. But it seems we may have to extend the default importer.
Query : Is this s feasible option to extend the 'GenericMultiThreadedImporter' to read files/folders from S3? Any comments/guidance would be great.
Read about transient S3 bucket in Nuxeo document. So using the S3 webui user can upload the files to S3 and Document says Nuxeo will automatically move this to its permanent S3 bucket. Any help on the second part how Nuxeo will pull from the transient bucket and move to the permanent S3 bucket.
Worst case- Using a lambda function copy the files in S3 to the drive accesible to the Nuxeo Server. But this will create two copies of the documents and double the gigs and costs.
Other other way to do this.
Appreciate any help/guidance. Thanks.
