games and code code it, play it

When I installed Ubuntu I couldn’t find any backup tool with features I wanted. I also wanted some exercise in Python so I wrote a backup tool and called it PySync. In meantime it became obvious to me that there is much better way to backup under Linux. As always with Linux, there really isn’t any full featured tool but with little time and knowledge you can modify everything to suit your needs. Combining rsync, bash script and cron, now I have backup system which does everything I need.

Features I wanted:

  • Daily backup
  • Only copy changed files
  • Write a log with list of changed files
  • Read a list of directories and synchronize them one by one
  • Way to exclude some files or folders

So here’s the procedure.

1. Write a script

The script contains a list of directories that will be synced. Log is created in a subfolder, one file for every day. I’ve put the script in ‘/usr/local/bin/backup-all’ file.

#!/bin/bash
# backup procedure by Hugo Riley, www.gamesandcode.com
BACKUP_LOG=/usr/local/bin/backup-log/backup-log-$(date +%Y%m%d)
EXCLUDE_LIST=/usr/local/bin/backup-exclude
echo "************************" > $BACKUP_LOG
echo "System backup with RSync" >> $BACKUP_LOG
date >> $BACKUP_LOG
echo "************************" >> $BACKUP_LOG
FILES=(
'/boot /media/backup/filesystem'
'/home /media/backup/filesystem'
'/opt /media/backup/filesystem'
'/media/data/Dokumenti /media/backup'
'/media/data/Music /media/backup'
'/media/data/Projects /media/backup'
)
ELEMENTS=${#FILES[@]}
for (( i=0;i<$ELEMENTS;i++)); do
folderpair=${FILES[${i}]}
rsync --delete --delete-excluded -av $folderpair --exclude-from=$EXCLUDE_LIST >> $BACKUP_LOG
done
#end of script

Script must be executable to work so it needs to be chmoded:

chmod +x backup-all

Files and folders can be excluded from backup. Just add some filters to backup-exclude file:
.trash
Trash
Cache
.thumbnails

2. Make a schedule

To enable daily backup, you need to add it to the Anacron. It can be modified in ‘/etc/anacrontab’. I wanted my backup to start every day, ten minutes after Linux is finished booting (login is not necessary). After the editing, my anacrontab file looks like this:

# /etc/anacrontab: configuration file for anacron
# See anacron(8) and anacrontab(5) for details.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# These replace cron's entries
1	5	cron.daily	 nice run-parts --report /etc/cron.daily
7	7	cron.weekly	 nice run-parts --report /etc/cron.weekly
1	10	SystemBackup	/usr/local/bin/backup-all
@monthly	15	cron.monthly nice run-parts --report /etc/cron.monthly

Category: Projects
Tags:

    4 Responses to “Linux system backup with RSync”

  1. Andrew Says:

    I have a question…
    with the line in /etc/anacrontab:

    1 10 SystemBackup /usr/local/bin/backup-all

    Does this mean the task is scheduled to take place daily, 10 minutes after the computer is first booted up?

  2. hugo Says:

    Yes, Andrew, you are correct. I didn’t make this clear enough, thanks for pointing this to me. I’ve edited the article so it’s clear now.

  3. alv Says:

    Have you checked the option –files-from ? It seems to me that using it with a file like “to_backup” (see below) as well as rsync’s logging features you could eliminate all your bash code and just leave the rsync invocation. Here my rsync command with some explanations.

    rsync -rihavz –files-from=to_backup –log-file rsync-log
    –rsync-path=”rsync –log-file=/home/me/rsync-log” –delete-after
    /home/me remote_machine:~

    what this does:

    -a is preserving permissions, and normally recursing (-r) when there
    is no –from-file. As we read our backup directories from the to_backup file, we also need
    -r recurse
    -h human file sizes (13.4M instead of so and so many bytes)
    -v verbose
    -z compress
    -i itemized list of stuff being transferred
    –log-file specifies a log file name locally
    –rsync-path path of remote rsync executable. We use this command in
    order to tell the remote rsync to also log the transfers (in case
    something fails in the client, the server knows which was the last
    transfer).
    –files-from specifies that the directories and concrete files to be
    backed up are to be read from to_backup
    –delete-after delete all files in server that have been
    moved/deleted in client, but do so after the transfer.
    remote_machine:~ dump the files on my home directory at the remote machine which is accessible via ssh with passphrase (if you don’t
    know what this means, check this out:
    http://sial.org/howto/openssh/publickey-auth/)

    the to_backup file simply contains a list of
    directories or single files to be recursively taken into account. If
    you wish to have md4 hashing for identification of changed files
    instead of timestamp and size or particular sets of excluded files,
    check the rsync man page, which is quite readable.

    Cheers,

    Álv.

  4. hugo Says:

    Thanks, I wasn’t aware of “-files-from” option. It looks like it could get the job done. (I suppose it’s quicker to write the bash script than to read the whole man page… :) )

Was this article helpful? Improve it with your comment.