topic Re: CIFS problems with large documents repository in Alfresco Archive

CIFS problems with large documents repository

icarrara — Sun, 25 Sep 2011 12:28:54 GMT

I installed Alfresco 3.4.d using Tomcat 6 and MySql 5.1 on a WIndows Server 2008 64 bit with 32 Gbyte of memory and fast SAS hard drives.The JVM that host Tomcat starts with the below configuration:-Xss1M-Xms16G-Xmx16G-XX:+UseConcMarkSweepGC-XX:+CMSIncrementalMode-XX:NewSize=1G-XX:MaxPermSize=512M-X

Re: CIFS problems with large documents repository

loftux — Mon, 26 Sep 2011 05:59:12 GMT

Do not backup Alfresco using CIFS, beacuse that is NOT a backup
http://wiki.alfresco.com/wiki/Backup_and_Restore

Also, if you want to copy files stored in Alfresco to another folder in Alfresco, consider using the Share web client, or the api. The reason is that a copy internally is just a db transaction (creating new nodes with pointers to the stored file), no files get moved around. So a copy or move of large amount of data is very quick this way.
With CIFS, you actually move the file content around.

Re: CIFS problems with large documents repository

icarrara — Mon, 26 Sep 2011 22:12:46 GMT

Hi! Thank you for your reply!

The fact is that I need to copy the CIFS content because I need to be able to restore only some folders or some files, not the entire repository.

About the copy operations inside CIFS folders, there are large amounti of users that use Alfresco only by CIFS folders interface, not by Web/Share interface.
Moreover, there are external systems and legacy procedures that feeds Alfresco repository only by CIFS folders… we can't forget the external environments.

For example, we have some IBM AS400 servers that store in Alfresco large amount of PDF files and the only method is to see CIFS folders….

So I need to have a stable CIFS interface for our CMS.

In your experience, how stable is the CIFS server provided by Alfresco ?

Thank you for more comments!

Ivano C.

Re: CIFS problems with large documents repository

loftux — Tue, 27 Sep 2011 07:45:00 GMT

OK, at least you are informed on how to do a backup of Alfresco :wink:

So, in your case, what could cause the issues? Your backup/copy solution, is it changing anything in the files, like trying to set an archive bit and thus the Alfresco latest update time stamp must be updated for every single file (not sure if that actually happens, just trying the idea). Can you check if the latest change time stamp gets updated? If so, is there anything you can change in your backup software settings?

Cifs isn't the best protocol (not because of also Alfresco, in the protocol itself), very chatty and not suitable to travel across routers and long distances. It also requires that all files be listed in a folder before you can start doing anything with any of the files. And for that, Alfresco needs to list all of them and evaluate permissions and find out its metadata. So in your case, do you have large amount of files in each folder? There is no exact number for a recommendation of number of files per folder, but I would say around 300. Then start creating subfolders to store your files in. That could improve things.

Also remember that Alfresco stores information in a database, something a normal cifs doesn't do. So there is a small overhead when storing files, the database has to do its work. So have you tuned your database settings anything? Like transactional cache size?
There is also indexing, the document needs to be indexed, and that adds compared to an ordinary file server save.

Do you have any rules on you folders, for example one that applies versioning? That also adds to the amount of work a cifs save needs to do.

My experience with cifs: Haven't tried your large copy scenario so really can't say if it is tunable with the above. For everyday work with Office documents, my experience with Community 3.4.x isn't the best because of Office rename shuffle when saving. In short, when you save an office document, it renames/saves with temp files, and alfresco needs to keep track of that. That often fail, and documents can get corrupted. Now with 4.0.a coming soon, there is lots of code changes to deal with this and other cifs stuff, so I'm optimistic that there are lots of improvements for CIFS there. Then you can use solr, so that indexing can be out of transaction and on separate server, and that may improve performance as well. So for 3.4.x, I try not to use cifs for anything except as a way to get multicopiers and such to have an interface to save to Alfresco, for 4.0.a, I'm a lot more optimistic.

Re: CIFS problems with large documents repository

icarrara — Tue, 27 Sep 2011 13:48:04 GMT

Hi!

You was very clear and your reply very complete … thank you!!

In definitive you are telling me that the actual 3.4.d is not good to substitude a normal Windows shared disk deployed on a server?

Alfresco is famous based on the fact that this scenario is a normal situation… right now I'm a bit surprised….
What is your advise in a scenario where users used to share Office's documents over a shared Windows disk, when you need to introduce Alfresco ?

Thank you veru much for your experience!

Ivano C.