topic Re: Need advice about backup/restore in Alfresco Archive

Need advice about backup/restore

zomurn — Wed, 25 Feb 2009 16:24:45 GMT

Hello,I'am was wondering how to proceed for backup/restore of the server in this scenario:1) The production version is working well.2) I backup the production version3) 1 week later, I update the custom model into which I add more metadata. Hence, I apply the default new metadata values to all docum

Re: Need advice about backup/restore

derek — Wed, 25 Feb 2009 17:43:57 GMT

Hi,
If you restored to before you installed the new model, then you should not have the new metadata anywhere in the data. I don't understand.
Regards

Re: Need advice about backup/restore

zomurn — Thu, 26 Feb 2009 09:12:00 GMT

Yes you're right. But I need to say to my customer (150 persons) that the job they've done during 1 week (injecting the thousands documents through CIFS) is lost, they need to redo their scan….impossible !
Hence I need to keep the "HEAD version" of alf_data + database

Re: Need advice about backup/restore

zaizi — Thu, 26 Feb 2009 09:16:22 GMT

There is no way you can keep the latest alf_data + database without the model changes. Because they make the changes.

Your best course of action is to work out what caused the crashed and try and fix it.

Re: Need advice about backup/restore

zomurn — Thu, 26 Feb 2009 09:29:54 GMT

OK thanks.
Fortunately, to deal with that, I have a pre production version which is the mirror of the production version (at least the lastest model, so not necessarily the latest alf_data+database ).
Hence I can test the deployment to test if the disaster might occurs.
One last question, is there only one "critical" file in alfresco : editing the "custom-model.xml" file ?
Normally, all other files can be edited without risk, isn't it ?

Thanks for your advice.

Re: Need advice about backup/restore

zaizi — Thu, 26 Feb 2009 10:49:50 GMT

If you mean by critical, files that modify the repository, there are a bunch of them.

If you have a pre-production servers, then all changes should be tested on them first. You should not be modifying directly on live. That doesn't just apply for Alfresco.

Re: Need advice about backup/restore

zomurn — Thu, 26 Feb 2009 11:02:56 GMT

Of course for pre production, I do the test of new functionalities in addition to check that alfresco server start up well.
Exactly, it is was I mean for critical files…I know only custom-model.xml for it, you afraid me….can give me some more files ?

Thanks.

Re: Need advice about backup/restore

derek — Thu, 26 Feb 2009 11:34:37 GMT

Hi,

We attempt to write the code so that any unrecognized types, aspects and properties (they're in the data but not in the model) are just handled normally. It is very likely, if you are running on the latest release, that the system will work fine. If not, identifying and cleaning up the data can be done directly on the database.

Your maximum backup interval is dictated by how much you can afford to lose. If you can't afford to lose a week when you restore, then you have to take backups more regularly. You can do nightly or hourly backups of the DB and then weekly backups of the filesystem if (a) you have a reliable filesystem and (b) you protect the content from the orphan cleanup for at least a week.

Re: Need advice about backup/restore

zomurn — Thu, 26 Feb 2009 12:40:59 GMT

Your maximum backup interval is dictated by how much you can afford to lose. If you can't afford to lose a week when you restore, then you have to take backups more regularly. You can do nightly or hourly backups of the DB and then weekly backups of the filesystem if (a) you have a reliable filesystem and (b) you protect the content from the orphan cleanup for at least a week.

Thanks very much for your participation…but answers call new questions which calls new answers and so on you know .

1) Can you explain point (b)

2) "How much you can afford to lose" : nothing, it's likely to be the answer of the customer .
Why backup the database more often ? Suppose I backup alf_data each monday and db each night. I have a crash before next monday….the db backup from tuesday to sunday is useless because newer than alf_data , isn't it ? The backup of the database must be older (= hot backup) or at same version (= cold backup)) of alf_data.

From this and from the fact I have to often backup (suppose each day) it would suppose I need to backup alf_data each day…it cost a lot in space requirement unless I do incremental backup (and I need to I suppose ). Moreover the customer as RAID 5 machine with hardware mirroring backup.
Personnaly if I have a problem to deploy in production, I would like not to fix nothing (like a crash of window). I prefer to restore the lastest backup which is sure, safe, fast (?) and postpone the next version later after identifying the problem (which can be time varying) on my post.

Re: Need advice about backup/restore

derek — Thu, 26 Feb 2009 13:28:42 GMT

but answers call new questions which calls new answers and so on you know

Yes. But at least I don't end up saying stuff you already know.

1) http://wiki.alfresco.com/wiki/Content_Store_Configuration#Content_Binaries.27_Lifecycle. Content (at the storage level) is only every created by the user - never modified. Content that has been orphaned is deleted by a background job. This doesn't clean out anything younger than 7 days, but this can be changed by overriding the bean from scheduled-jobs-context.xml. It follows that, during that protected period, you can backup and restore your db seamlessly. The general ordering rules governing hot backup still apply: You can backup the DB and then the filesystem; the time between the two (DB and FS) must not exceed the time that orphaned content is protected. So you can run regular DB backups and only run irregular FS backups. You can also only backup newer content by targeting subdirectories of the content store (since they're time-located).

2) Since the FS is running on a RAID system, it has redundancy and doesn't need to be backed up that much. Once a week for RAID is probably OK. To protect yourself against software and user-related issues (i.e. unpredictable reliability), you'll need to backup the DB a lot more often; this is especially true after an upgrade or modification of the software. If you followed point 1, you'll take very frequent hot backups of the DB followed by hot backups of the filesystem. You will therefore always have a DB that is less than 7 days old when you copy the content.

Regards

Re: Need advice about backup/restore

derek — Thu, 26 Feb 2009 14:04:20 GMT

I forgot to mention, that you can try the following:

Re: Need advice about backup/restore

zomurn — Mon, 02 Mar 2009 15:37:40 GMT

Is it "avmOrphanReaperJob" in scheduled-jobs-context to set to 7 days for parameter "repeatIntervalMinutes" or "protectDays" in content-services-context.
For example, I crashed my dev post right now and alfresco refuse to start…only by recompiling my jar file (edited the java code only and .xml spring beans file) and I'd like to simulate what to do if it happens like in prod without knowing why it crashes…what do you suggest to fix the problem of "a previous schema upgrade failed…"

Thanks

Re: Need advice about backup/restore

derek — Mon, 02 Mar 2009 16:01:50 GMT

It's protectDays (as per Wiki page referenced).

If your system crashes during install or upgrade, then you need to reset or restore your DB and start again. Clearly, Alfresco can't start up on a half-baked schema.

I repeat: You should not be modifying your config within the jar files. Use the proper override mechanisms.

Re: Need advice about backup/restore

zomurn — Mon, 02 Mar 2009 17:04:24 GMT

But inside my jar file. The only xml files are spring beans and the module-context.xml which import only these xml spring files.
My model is kept in alfresco/classes/extension in the WEB-INF directory (the WEBAPP).
Honestly, I really didn't know why my post has crashed…I restored a previous version db+alf_data and all was recovered.

Re: Need advice about backup/restore

derek — Wed, 04 Mar 2009 20:51:43 GMT

Well, the "crash" would have to be investigated separately (you could post more details). Do you want to know more about the backup strategy?

Re: Need advice about backup/restore

zomurn — Fri, 06 Mar 2009 09:10:49 GMT

Of course I want … even if my customer do a copy/paste each day on a backup server.
Otherwise, another time, this morning, I start my computer and run my tests one more time….I have got for the first time on my test repository "A previous schema upgrade failed" where as I only code test and run them on it. I really don't understand why my repository has crashed.

Whole stack trace :

10:06:12,817 INFO  [alfresco.mbeans.VirtServerRegistry] Creating VirtServerRegistry MBean
10:06:13,863 WARN  [remoting.rmi.RmiRegistryFactoryBean] Could not detect RMI registry - creating new one
10:06:14,114 INFO  [alfresco.mbeans.VirtServerRegistry] Created JMX serverConnector
10:06:14,161 DEBUG [alfresco.module.msc] preRegister called. Server=com.sun.jmx.mbeanserver.JmxMBeanServer@1f784d7, name=log4j:logger=org.alfresco.module.msc
10:06:16,501 INFO  [domain.schema.SchemaBootstrap] Schema managed by database dialect org.hibernate.dialect.MySQLInnoDBDialect.
10:07:16,624 ERROR [domain.schema.SchemaBootstrap] Schema auto-update failed
org.alfresco.error.AlfrescoRuntimeException: A previous schema upgrade failed.  Revert to the original database before attempting the upgrade again.
   at org.alfresco.error.AlfrescoRuntimeException.create(AlfrescoRuntimeException.java:46)
   at org.alfresco.repo.domain.schema.SchemaBootstrap.setBootstrapStarted(SchemaBootstrap.java:510)
   at org.alfresco.repo.domain.schema.SchemaBootstrap.onBootstrap(SchemaBootstrap.java:949)
   at org.alfresco.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:62)
   at org.springframework.context.event.SimpleApplicationEventMulticaster$1.run(SimpleApplicationEventMulticaster.java:77)
   at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
   at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:75)
   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:241)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:349)
   at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:92)
   at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:77)
   at org.alfresco.util.ApplicationContextHelper.getApplicationContext(ApplicationContextHelper.java:55)
   at org.alfresco.module.msc.common.test.MscAbstractTestCase.<clinit>(MscAbstractTestCase.java:36)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
   at junit.framework.TestSuite.createTest(TestSuite.java:131)
   at junit.framework.TestSuite.addTestMethod(TestSuite.java:114)
   at junit.framework.TestSuite.<init>(TestSuite.java:75)
   at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestLoader.getTest(JUnit3TestLoader.java:102)
   at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestLoader.loadTests(JUnit3TestLoader.java:59)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:445)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
java.lang.ExceptionInInitializerError
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
   at junit.framework.TestSuite.createTest(TestSuite.java:131)
   at junit.framework.TestSuite.addTestMethod(TestSuite.java:114)
   at junit.framework.TestSuite.<init>(TestSuite.java:75)
   at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestLoader.getTest(JUnit3TestLoader.java:102)
   at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestLoader.loadTests(JUnit3TestLoader.java:59)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:445)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
   at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
Caused by: org.alfresco.error.AlfrescoRuntimeException: Schema auto-update failed
   at org.alfresco.repo.domain.schema.SchemaBootstrap.onBootstrap(SchemaBootstrap.java:1002)
   at org.alfresco.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:62)
   at org.springframework.context.event.SimpleApplicationEventMulticaster$1.run(SimpleApplicationEventMulticaster.java:77)
   at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
   at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:75)
   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:241)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:349)
   at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:92)
   at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:77)
   at org.alfresco.util.ApplicationContextHelper.getApplicationContext(ApplicationContextHelper.java:55)
   at org.alfresco.module.msc.common.test.MscAbstractTestCase.<clinit>(MscAbstractTestCase.java:36)
   … 13 more
Caused by: org.alfresco.error.AlfrescoRuntimeException: A previous schema upgrade failed.  Revert to the original database before attempting the upgrade again.
   at org.alfresco.error.AlfrescoRuntimeException.create(AlfrescoRuntimeException.java:46)
   at org.alfresco.repo.domain.schema.SchemaBootstrap.setBootstrapStarted(SchemaBootstrap.java:510)
   at org.alfresco.repo.domain.schema.SchemaBootstrap.onBootstrap(SchemaBootstrap.java:949)
   … 23 more
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Need advice about backup/restore

derek — Mon, 09 Mar 2009 10:46:29 GMT

Hi,

Revert to the original database before attempting the upgrade again.

At some or other point, you attempted an upgrade (or installation) that either failed or was terminated midway. You need to revert to a good copy of the data. You will not be able to recover the server with any confidence. When you upgrade Alfresco, you should be prepared to have schema updates applied and have the appropriate backups beforehand.
Regards

Re: Need advice about backup/restore

archana_bonkanp — Tue, 10 Mar 2009 06:34:35 GMT

Hi,

I have some doubts related to backup and recovery.
We are taking the backup of Alfresco application daily.
the steps followed are
1)we will stop the server
2)take the db backup
3)take the alf_data backup
4)then restart the server

but its taking lot of time in hours.
can you olease suggest us any other means of taking backup,since restoratin is also taking huge amount of time.

Re: Need advice about backup/restore

derek — Tue, 10 Mar 2009 11:57:24 GMT

Hi,
You can do a hot backup i.e. backup with the system still running.
1. Wait until the lucene index backup is done i.e. set that to happen a bit before your backup starts
2. Do a transactionally-safe backup of the DB
3. Backup the content (content stores and backup lucene indexes). NOT the live Lucene indexes.
4. Clean out the contentstore.deleted

Restoring your DB and Lucene indexes is one thing, but you will need to run the system with index.recovery.mode=AUTO as (you may have spotted it) the Lucene indexes could be out of date.

The contentstore is partitioned by date ranges. You can, if you already have year 2008 backed up, for example, only backup current 2009.

Re: Need advice about backup/restore

archana_bonkanp — Wed, 11 Mar 2009 05:26:00 GMT

Hi,

The content is stored datewise only (2007,2008,2009).
but if any user deletes the data which is uploaded in 2008 and if i dont take 2008 bckup then there will not be no problem right.