Here’s two backup scripts that I wrote, one for full backup and one for incremental backup. The way it works is that whenever the full backup script is run it creates a snapshot file (basically a hash) of the files and folders it backs up. Then when the incremental backup runs it checks against that file to see which files have been changed and only backs up those files. So as an example, if you run full backup every sunday and then incremental backup every day, then tuesday’s backup will include both the files changed on monday and on tuesday (essentially all files since the last full backup). This means that to restore a backup you only need to untar the latest full and latest incremental backup.

Read more about that here in the tar manual:
http://www.gnu.org/software/tar/manual/html_node/Incremental-Dumps.html#SEC94

The scripts are run as root, but I created a separate user called backupuser on both machines that will handle the ssh connection. You can read more about setting up SSH authentication using keypairs here: http://jstr.se/archives/ssh-login-with-rsa-key-pairs/
Just make sure you leave the passphrase empty if you want to use SSH login with keys in a script.

There are probably tons of ways on how you can improve on this script. But hopefully it helps someone! Feel free to leave a comment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/bin/bash
#Full backup script
 
SERVER="backupserver.example.com"
MYSQLPASSWD=""
MYSQLTEMP="/tmp/mysql.all-databases.sql.gz"
FILESTOBACKUP="/home /share /var/www $MYSQLTEMP"
TEMPBACKUPFOLDER="/tmp"
ARCHIVENAME="`date +'%Y-%m-%d-full.tar.gz'`"
SNAPSHOT="/var/log/backup/snapshot"
LOGFOLDER="/var/log/backup"
 
timeStamp() {
while read line; do
    echo "$(date): ${line}" >> $LOGFOLDER/backup.log
done
};
 
if [ ! -d $LOGFOLDER ]; then
    mkdir -p $LOGFOLDER
fi
 
if [ -f $SNAPSHOT ]; then
    mv $SNAPSHOT ${SNAPSHOT}`date +'%Y-%m-%d'`
 
else
    echo "$(date): $SNAPSHOT was not found, creating it.." >> $LOGFOLDER/backup.log
fi
 
mysqldump -h localhost -u root -p$MYSQLPASSWD -A | gzip > $MYSQLTEMP
nice -n 15 tar --create --gzip --listed-incremental=$SNAPSHOT --file=$TEMPBACKUPFOLDER/$ARCHIVENAME $FILESTOBACKUP 2> >(timeStamp)
 
rm $MYSQLTEMP
 
su -l backupuser -c "scp $TEMPBACKUPFOLDER/$ARCHIVENAME $SERVER:~/" 1> /dev/null 2> >(timeStamp)
 
if [ $? == 0 ]; then
    echo  "$(date): Backup of $ARCHIVENAME to $SERVER succeeded" >> $LOGFOLDER/backup.log
    rm $TEMPBACKUPFOLDER/$ARCHIVENAME
 
else
    echo "$(date): !! ERROR !! Failed to transmit $ARCHIVENAME to $SERVER !!" >> $LOGFOLDER/backup.log
fi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#!/bin/bash
#Incremental backup script
 
 
SERVER="backupserver.example.com"
MYSQLPASSWD=""
MYSQLTEMP="/tmp/mysql.all-databases.sql.gz"
FILESTOBACKUP="/home /share /var/www $MYSQLTEMP"
TEMPBACKUPFOLDER="/tmp"
ARCHIVENAME="`date +'%Y-%m-%d-%H%M%S-incremental.tar.gz'`"
SNAPSHOT="/var/log/backup/snapshot"
LOGFOLDER="/var/log/backup"
PATHTOFULLBACKUP="/root/fullback.sh"
 
timeStamp() {
while read line; do
    echo "$(date): ${line}" >> $LOGFOLDER/backup.log
done
};
 
if [ ! -d $LOGFOLDER ]; then
    mkdir -p $LOGFOLDER
fi
 
if [ -f $SNAPSHOT ]; then
cp $SNAPSHOT $SNAPSHOT.inc
mysqldump -h localhost -u root -p$MYSQLPASSWD -A | gzip > $MYSQLTEMP
nice -n 15 tar --create --gzip --listed-incremental=$SNAPSHOT.inc --file=$TEMPBACKUPFOLDER/$ARCHIVENAME $FILESTOBACKUP 2> >(timeStamp)
rm $MYSQLTEMP
su -l backupuser -c "scp $TEMPBACKUPFOLDER/$ARCHIVENAME $SERVER:~/" 1> /dev/null 2> >(timeStamp)
 
    if [ $? == 0 ]; then
        echo  "$(date): Backup of $ARCHIVENAME to $SERVER succeeded" >> $LOGFOLDER/backup.log
        rm $TEMPBACKUPFOLDER/$ARCHIVENAME
    else
        echo "$(date): !! ERROR !! Failed to transmit $ARCHIVENAME to $SERVER !!" >> $LOGFOLDER/backup.log
    fi
else
    echo "!! ERROR !! No snapshot file found at $SNAPSHOT, performing full backup instead.."
    $PATHTOFULLBACKUP
fi

Post filed under Linux, Server Administration.

2 Comments

  1. Hi,

    this post was really helpful for me. Many thanks !

    But it seem’s that there’s a small bug in the incremental script.
    When you make a copy of the full snap, if there’s already an incremental snap, it’ll not be copied (you probably have to force or drop the previous incr snap before).

    Regards

  2. I love what you guys tend to be up too. This sort of clever work and
    reporting! Keep up the terrific works guys I’ve included you guys to blogroll.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="" highlight="">