topic Re: Other File Systems with Alfresco - HadoopFS? in Alfresco Archive

Other File Systems with Alfresco - HadoopFS?

ashpal19 — Wed, 26 Feb 2014 22:42:36 GMT

Hello,As per my understanding Alfresco currently uses the NTFS for storing of the files. Below are my questions - 1) if we can integrate Alfresco with the Hadoop File System for storage of the actual documents? 2) If not HadoopFS, can Alfresco work with(integrate) with any other system?Thank you for

Re: Other File Systems with Alfresco - HadoopFS?

mrogers — Thu, 27 Feb 2014 11:53:00 GMT

No its not specific to NTFS but alfresco uses the OS filesystem. You can use any system that mounts or looks like a filesystem.

Its also possible to integrate with other content stores, for example I think Alfresco Enterprise comes with an Amazon S3 store.

I can't see a hadoop store in the alfresco add ons, sorry.

Re: Other File Systems with Alfresco - HadoopFS?

nickburch — Sun, 29 Jun 2014 17:59:35 GMT

Writing your own Content Store implementation isn't too hard, so producing a HDFS backed one wouldn't take that much time.

Your big issue is that HDFS normally uses quite a large block size by default (IIRC 64mb), so if you're storing small files you'll either want to implement your own way of putting multiple files into one HDFS block, or accept lots of wasted space. Dropping the block size isn't generally recommended, as that puts too much pressure on the name node. If you do store multiple small files in one HDFS block, think carefully about how you do it to ensure that other tools you might run across the files (eg Map Reduce jobs) can easily work with them.