topic Re: Clarification on Backup file paths in Alfresco Archive

Clarification on Backup file paths

eswbitto — Tue, 16 Jul 2013 18:21:14 GMT

I'm in the testing phase of trying to backup both file content and database and I need some clarity on the database. I've followed the wiki on doing hot/cold backups, but when I look at my file path I have two postgresql directories./opt/alfresco/alf_data/postgresql/opt/alfresco/postgresqlwhich one

Re: Clarification on Backup file paths

fcorti — Tue, 16 Jul 2013 18:51:01 GMT

Hi ESWBitto,

As described in the wiki, the correct sequence to backup alfresco is: backup of the db, backup of the documents and indexes (I suppose that both are stored in the alf_data folder).
The DB backup is usually done using the backup tools of your database: in your case postgresql.

I imagine there is something unusual with your installation because you have two different postgres folders, probably one is not used or is used for different database respect to the Alfresco's one.
But in your case, you can ignore it using the 'pg_dump -Ft -b alfresco > db_backup.sql' command.

The wiki is definitely clear but if you want to see a practical example I suggest you this <a href="https://francescocorti.wordpress.com/2013/02/06/alfresco-backup-script/">link</a>.

Hope this help you.

Re: Clarification on Backup file paths

eswbitto — Tue, 16 Jul 2013 19:01:39 GMT

I'm not sure why its creating two different locations. I'm using the 4.2.c bin file to install alfresco.

Re: Clarification on Backup file paths

eswbitto — Tue, 16 Jul 2013 19:22:00 GMT

Also,

Thanks for making this script it really does help out a lot. When I run it though I get an error on line 27.


./alfrescoBackup.sh: line 27: /opt/postgresql/bin/pg_dump: No such file or directory
‍‍‍

Any ideas? I looked at /opt/alfresco/postgresql/bin and the file is there according to the warning. Weird.

*edit* ah I just looked at it….its looking for /opt/postgresql instead of /opt/alfresco/postgresql

Re: Clarification on Backup file paths

eswbitto — Tue, 16 Jul 2013 20:13:44 GMT

I am getting this message though.

pg_dump.bin: [archiver (db)] connection to database "alfresco" failed: could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?

My guess is when the script stops the service to run the dump it can't find it because its not running.

Re: Clarification on Backup file paths

fcorti — Wed, 17 Jul 2013 08:09:09 GMT

Hi ESWBitto,

Sincerely I'm not sure because we don't usually use the bundle installation but it's easy to check: you can stop the service manually and try to execute the db connection or exportation.
In this case you can remove the stopping command but remember that in this case it's a hot backup.

Let us know.

Re: Clarification on Backup file paths

bopolissimus — Wed, 17 Jul 2013 12:19:00 GMT

/opt/alfresco/alf_data/postgresql is where the data is.
/opt/alfresco/postgresql is where the postgresql software is.

if you want to do a cold backup, just stop alfresco (/opt/alfresco/alfresco.sh stop) and then backup the whole /opt/alfresco directory (or, for a minor space saving), /opt/alfresco/alf_data.

if you want to do a hot backup, alfresco would still be running. let's say you're going to put the backup in /var/backups/alfresco (this is what we do).

sudo mkdir /var/backups/alfresco
sudo chown [user].[group] /var/backups/alfresco #set user and group to whomever runs the alfresco process.

# do an initial rsync, minimize time for later rsync.
sudo rsync -avv -P –delete /opt/alfresco/alf_data /var/backups/alfresco/alf_data

cd /opt/alfresco/postgresql/bin
./pg_dump -ORc -h localhost -U [alfresco_user_name] [alfresco_db_name] > /opt/alfresco/alf_data/alfresco-db-backup.sql

# do a refresh rsync, faster than the first one.
sudo rsync -avv -P –delete /opt/alfresco/alf_data /var/backups/alfresco/alf_data

that should do the trick. note: you'll need to inject the correct values wherever I have [SOMETHING_IN_SQUARE_BRACKETS]

there's another way to do a cold backup (stop alfresco and then start postgres and pg_dump that). but I can't describe the procedure since I've forgotten what it is, and I don't need perform that backup procedure since we don't stop alfresco, just always do hot backups.

Re: Clarification on Backup file paths

eswbitto — Wed, 17 Jul 2013 15:35:06 GMT

Thanks Bopolissimus,

I will try and test that and see how it goes. I initially did something similar. I stopped alfresco then copied the alf_data directory to a different location. Started Alfresco. Went in and deleted a test document I had been working with. Then tried to restore by stopping alfresco again. Replacing the alf_data directory and then starting alfresco. It didn't like that. I ended up having to re-install the whole thing again.

I'm going to try the steps you outlined and I'll let you know how it goes.

Re: Clarification on Backup file paths

eswbitto — Wed, 17 Jul 2013 15:49:25 GMT

@ fcorti

Ok think I figured out where I went wrong. I commented out the stopping command and tried to run the script. When I did that it asked for a password. So I gave it my database password (my assumption of what it wanted) and it worked. So that is where I went wrong. I looked through the shell script again and there is a perimeter that I missed on putting in the password.

Re: Clarification on Backup file paths

eswbitto — Wed, 17 Jul 2013 21:02:00 GMT

@ fcorti
ok so I have implemented your backup strategy to see how it functions for my environment. Here are the steps that I did. I have a question at the end so hang tight.

1) I used the instructions from HERE
2) I figured out that I was not creating the .pgpass file and giving the right attributes (0660) that it calls for. Got that done and the script works for me. It created the tgz file so good to go.
3) In testing…I deleted a file from a testing site that I wanted to restore.
4) Moving on to test restoring: So I followed the instructions from HERE To a degree.
   A. As a precaution I moved the alf_data directory to a different location before doing any steps.
   B. I stopped the service…skipped down to the part for extracting the tgz file.
   C. In doing so when I tried to extract using


gzip -dc 2013-07-17_12-02_alfresco_dump.tgz | tar xf -
‍‍‍

I would get two additional tar files:
2013-07-17_12-02_alfresco_postgresql.tar
2013-07-17_12-02_alfresco_data.tgz

When I extracted the data.tgz file I get the whole alf_data directory back.
When I extract the postgresql.tar file I get a bunch of dat files.

Now here is my question….are the dat files in fact the lucene index files or are they something different.

The reason I ask is I removed the dat files altogether and just relied on the first file to see if I did restore the data. In which case I was successful. Any feedback would be helpful.

The other issue I have is the backup strategy doesn't allow for incremental restoration. Meaning…If I have to create a restore point of the previous day for one file. Then any content added after that time frame would be lost.

I'm open to any ways on how to restore incremental data.

Re: Clarification on Backup file paths

bopolissimus — Thu, 18 Jul 2013 01:28:36 GMT

Yes, as you've found out, you can't do that. The reason is, the state of alfresco is the union of alf_data contents AND database contents at the same point in time (or close enough in time that no content changes have happened between taking a backup of alf_data and the database).

When you delete the test document and then restore the old alf_data, alfresco won't start because it'll notice that your database and alf_data are out of sync. There is content in the old alf_data which the database doesn't know about (it doesn't know it's name, for instance, or where its versions are, or who edited it last, all that metadata is in the database).

This is why, when taking a backup, you keep the database dump and the alf_data together. so that when you restore, you restore both. restoring just one breaks alfresco since alfresco will see that data and database are out of sync.

Re: Clarification on Backup file paths

fcorti — Thu, 18 Jul 2013 05:38:47 GMT

Hi ESWBitto,

Reading your comment I suppose you are trying to compact your backup as much as possibile… that's correct, of course.
Incremental backup in Alfresco is possible if we understand the backup strategy.

Trying to avoid to reinvent the wheel I suggest you <a href="http://forums.alfresco.com/forum/general/non-technical-alfresco-discussion/alfrtesco-backup-size-and-compression-issues">this post</a> from MRogers (Alfresco Engineer).
In my personal opinion this strategy sounds a little bit tricky compared with other ECM leaders. 😉

About .dat files, please, confirm us they are stored in other sub folders than contentstore.
If yes, as I'm quite sure, they are not your documents (the documents are stored as bin files).
To understand better the use of the alf_data subfolders I suggest you to read <a href="https://forums.alfresco.com/forum/installation-upgrades-configuration-integration/configuration/alfdata-question-12142012-0807">here</a>.

Hope this help you (and thank you for the request).