Here’s two backup scripts that I wrote, one for full backup and one for incremental backup. The way it works is that whenever the full backup script is run it creates a snapshot file (basically a hash) of the files and folders it backs up. Then when the incremental backup runs it checks against that file to see which files have been changed and only backs up those files. So as an example, if you run full backup every sunday and then incremental backup every day, then tuesday’s backup will include both the files changed on monday and on tuesday (essentially all files since the last full backup). This means that to restore a backup you only need to untar the latest full and latest incremental backup.
Read more about that here in the tar manual:
http://www.gnu.org/software/tar/manual/html_node/Incremental-Dumps.html#SEC94
The scripts are run as root, but I created a separate user called backupuser on both machines that will handle the ssh connection. You can read more about setting up SSH authentication using keypairs here: http://jstr.se/archives/ssh-login-with-rsa-key-pairs/
Just make sure you leave the passphrase empty if you want to use SSH login with keys in a script.
There are probably tons of ways on how you can improve on this script. But hopefully it helps someone! Feel free to leave a comment.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | #!/bin/bash #Full backup script SERVER="backupserver.example.com" MYSQLPASSWD="" MYSQLTEMP="/tmp/mysql.all-databases.sql.gz" FILESTOBACKUP="/home /share /var/www $MYSQLTEMP" TEMPBACKUPFOLDER="/tmp" ARCHIVENAME="`date +'%Y-%m-%d-full.tar.gz'`" SNAPSHOT="/var/log/backup/snapshot" LOGFOLDER="/var/log/backup" timeStamp() { while read line; do echo "$(date): ${line}" >> $LOGFOLDER/backup.log done }; if [ ! -d $LOGFOLDER ]; then mkdir -p $LOGFOLDER fi if [ -f $SNAPSHOT ]; then mv $SNAPSHOT ${SNAPSHOT}`date +'%Y-%m-%d'` else echo "$(date): $SNAPSHOT was not found, creating it.." >> $LOGFOLDER/backup.log fi mysqldump -h localhost -u root -p$MYSQLPASSWD -A | gzip > $MYSQLTEMP nice -n 15 tar --create --gzip --listed-incremental=$SNAPSHOT --file=$TEMPBACKUPFOLDER/$ARCHIVENAME $FILESTOBACKUP 2> >(timeStamp) rm $MYSQLTEMP su -l backupuser -c "scp $TEMPBACKUPFOLDER/$ARCHIVENAME $SERVER:~/" 1> /dev/null 2> >(timeStamp) if [ $? == 0 ]; then echo "$(date): Backup of $ARCHIVENAME to $SERVER succeeded" >> $LOGFOLDER/backup.log rm $TEMPBACKUPFOLDER/$ARCHIVENAME else echo "$(date): !! ERROR !! Failed to transmit $ARCHIVENAME to $SERVER !!" >> $LOGFOLDER/backup.log fi |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | #!/bin/bash #Incremental backup script SERVER="backupserver.example.com" MYSQLPASSWD="" MYSQLTEMP="/tmp/mysql.all-databases.sql.gz" FILESTOBACKUP="/home /share /var/www $MYSQLTEMP" TEMPBACKUPFOLDER="/tmp" ARCHIVENAME="`date +'%Y-%m-%d-%H%M%S-incremental.tar.gz'`" SNAPSHOT="/var/log/backup/snapshot" LOGFOLDER="/var/log/backup" PATHTOFULLBACKUP="/root/fullback.sh" timeStamp() { while read line; do echo "$(date): ${line}" >> $LOGFOLDER/backup.log done }; if [ ! -d $LOGFOLDER ]; then mkdir -p $LOGFOLDER fi if [ -f $SNAPSHOT ]; then cp $SNAPSHOT $SNAPSHOT.inc mysqldump -h localhost -u root -p$MYSQLPASSWD -A | gzip > $MYSQLTEMP nice -n 15 tar --create --gzip --listed-incremental=$SNAPSHOT.inc --file=$TEMPBACKUPFOLDER/$ARCHIVENAME $FILESTOBACKUP 2> >(timeStamp) rm $MYSQLTEMP su -l backupuser -c "scp $TEMPBACKUPFOLDER/$ARCHIVENAME $SERVER:~/" 1> /dev/null 2> >(timeStamp) if [ $? == 0 ]; then echo "$(date): Backup of $ARCHIVENAME to $SERVER succeeded" >> $LOGFOLDER/backup.log rm $TEMPBACKUPFOLDER/$ARCHIVENAME else echo "$(date): !! ERROR !! Failed to transmit $ARCHIVENAME to $SERVER !!" >> $LOGFOLDER/backup.log fi else echo "!! ERROR !! No snapshot file found at $SNAPSHOT, performing full backup instead.." $PATHTOFULLBACKUP fi |
Davide says:
November 4, 2011 at 11:08 am /
Hi,
this post was really helpful for me. Many thanks !
But it seem’s that there’s a small bug in the incremental script.
When you make a copy of the full snap, if there’s already an incremental snap, it’ll not be copied (you probably have to force or drop the previous incr snap before).
Regards