Alfresco - Back it on up

[rdanner]

If alfresco is to play neatly in the IT world I would think it is going to need to handle some management concerns.

When using alfresco as a/to replace "shared drive".

It is not economical to think that one should backup an export of alfresco on any kind of frequent basis if the volume of data is large.

Backing up just the alfresco file system doesn’t help because all the Meta data is in the database.  It could be somewhat painful to keep the file system and the database backups consistent.

It is possible that alfresco will need to play nicely in the backup industry. As well as providing hooks for document lifecycle.

What are the thoughts in this space?  What are the thoughts around how alfresco should be managed from systems prospective?

[derek]

Hi,

The backup of the system has 3 requirements:

Database: Any decent database has a replication / backup mechanism.  We looked at CJDBC to replicate the databases, but had issues with the BLOB support - but also it would be replacing perfectly good (fast) replication mechanisms provided by enterprise databases.  Which database are you using or planning to use?

Content: Backing up the filesystem can be performed regularly using RSync or any other type of filesystem replication or backup mechanism.  There are mechanisms to synchronously replicate the content store, but these should only really be used in a clustered system.  Normal filesystem replication is adequate for server replication, but in-house asynchronous replication will be available eventually.

Indexes: These can be rebuilt from scratch or incrementally as they are derived data.

We have made some changes that will support the case where content lags the metadata; if you have a system crash and the database metadata is more recent than the content, then the system will function but produce appropriate errors to highlight missing content without preventing metadata manipulation.  The reindexing will also allow a special search to pull back all content that didn't make it into the backup.

Regards

[rdanner]

Well my question  with this is,

There are two parts of a whole, the content store, and the metadata store.  The metadata store brings meaning to the content store. 

My understanding (may be wrong) is that the content store is useless without appropriate metadata to tell me what *.bin file is what.  Without the metadata I have a nice big collection of bits.

From a management experience I am not going to want to back these up via separate mechanism then restore them together and hope to God they are congruent.

I would prefer to be able to do wholesale and incremental exports from the alfresco repository which include both information and meta information.

One concept I think alfresco is attempting to cover is the idea that it can be used to replace a file share system.  On a file share system I backup individual items (that have no meta information) and when a user makes an error  like an accidental delete I can bring that file back from backup. 

I can also tell the system to begin to migrate scarcely accessed information to secondary or backup storage.  The information is replaced with a pointer and when the user accesses the pointer the information is recalled from secondary storage.

I don’t want to do a full recovery of the system because some user deleted a single file. 

Even if I could recall the file from backup, I would be missing the metadata so the file would be useless.  Heck without the metadata I cant even identify which file to bring back.


incremental export would be great. If I have incremental I can efficiently backup my system.  A full export once a month, an incremental each night in between.

the current export supports the ability to export specific nodes.  This could be used to support integration with a document lifecycle system. I could export scarcely accessed file/metadata pairs and move the information elsewhere leaving the metadata and additional information pertaining to retrieval of the actual document.

I guess document deletes could be made logical for some period of time before actual delete operations occur.

I guess we could add an aspect to the content which allows us to know when the last export of a given name was.  That way a content item could participate in many different incremental exports.

[rdanner]

Has anyone done a succesful export and import?  I had a successful export but the import has failed.

The following error has occurred:nimator:
Failed to parse view at line 604; column 57
org.alfresco.service.cmr.view.ImporterException: Failed to parse view at line 604; column 57
-rwxrwxrat org.alfresco.repo.importer.view.ViewParser.parse(ViewParser.java:134)
-rwxrwxrat org.alfresco.repo.importer.ImporterComponent.performImport(ImporterComponent.java:240)
-rwxrwxrat org.alfresco.repo.importer.ImporterComponent.importView(ImporterComponent.java:146)
drwxrwxrat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
-rwxrwxrat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
-rwxrwxrat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
-rwxrwxrat java.lang.reflect.Method.invoke(Method.java:585)l
-rwxrwxrat org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:287)
-rwxrwxrat org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:155)
drwxrwxrat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:122)
-rwxrwxrat org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:114)
-rwxrwxrat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at org.alfresco.repo.security.permissions.impl.AlwaysProceedMethodInterceptor.invoke(AlwaysProceedMethodInterceptor.java:32)
./java/jat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
total 88at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:36)
-rwxrwxrat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
-rwxrwxrat org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:57)
-rwxrwxrat org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
-rwxrwxrat org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:174)
-rwxrwxrat $Proxy32.importView(Unknown Source) 4.au
-rwxrwxrat org.alfresco.tools.Import.execute(Import.java:202)
-rwxrwxrat org.alfresco.tools.Tool.start(Tool.java:148)
-rwxrwxrat org.alfresco.tools.Import.main(Import.java:58)
Caused by: org.alfresco.service.cmr.action.ActionServiceException: Unable to retrieve the saved rule folder reference.
-rwxrwxrat org.alfresco.repo.rule.RuleServiceImpl.getSavedRuleFolderRef(RuleServiceImpl.java:152)
-rwxrwxrat org.alfresco.repo.rule.RuleServiceImpl.getRules(RuleServiceImpl.java:255)
        at org.alfresco.repo.rule.RuleServiceImpl.getRules(RuleServiceImpl.java:213)
./java/jat org.alfresco.repo.rule.RuleServiceImpl.hasRules(RuleServiceImpl.java:205)
total 12at org.alfresco.repo.rule.RuleTypeImpl.triggerRuleType(RuleTypeImpl.java:111)
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

My export is not totally simple.  I have several simple workflow and custom rules applied to the repository.

I haven't looked into the xml to see what the thing is complaining about yet but I thought I should get something up here and find out how well this is working for everyone.

[davidc]

Hi,

This is a known problem and a fix is currently being tested.