Federated Repository / Archival

[unknown-user]

Hi All,

We have a huge production repository with close to 7-8mn documents and the repository is growing in a fast pace with about 10K documents a week. Some of the MySQL tables (alf_node_properties) have close to a billion records in it. We have got pretty good servers, hence we haven't faced any performance issues so far.

But we are concerned about the growth of the MySQL DB and thinking of implementing some kind of custom Archival process. The idea is to move any document that is older than 6months to another Alfresco repository and keep the primary repository light and fast. We want our documents in the Archive server to be searchable from the primary server as well. We understand that through OpenSearch integration, this can be achieved but the challenge is that our clients access the repository through Web Service API's and custom UI's.

Is there any way to get this Federated Repository capability through Web Service/JavaScript API's. We are using 3.4.4.

Also, any other ideas on Archival is appreciated.

Thanks
Jerry

[mrogers]

With respect - that's not a "huge" repository.  

By all means implement an archival process if it makes sense for business reasons but at that point you may want to consider different APIs.

[unknown-user]

Thanks. Yes, we do understand that Alfresco can handle bigger repositories than this and as I mentioned we havent faced any performance issues so far.  But the MySQL table sizes is what is concerning us and that's what is leading us to the thoughts of implementing an Archival mechanism.

Can you give some inputs on the "different API's" that you mentioned? Were you referring to TransferService API's, if so, we are planning to use that to transfer the documents to Archive repository. But the problem is how to make the content in two repositories (Live & Archive) searchable for the user.

Thanks again for the quick response.

[mrogers]

If its just search you are worried about then once content is transferred to your archive system the logic is "search the primary system",  "search the archive system(s)", combine the results.     You may be able to do that in your own code or even have two separate web pages.   As and when alfresco gains sharding and federation for its solr indexes that's very crudely how it will work.

Transfer is not really intended to be used for archive purposes, but I guess it would work.

However once you split the properties and content between two different repositories then some things become harder and its two systems to manage rather than one.