March 28th, 2008
When I installed Ubuntu I couldn’t find any backup tool with features I wanted. I also wanted some exercise in Python so I wrote a backup tool and called it PySync. In meantime it became obvious to me that there is much better way to backup under Linux. As always with Linux, there really isn’t any full featured tool but with little time and knowledge you can modify everything to suit your needs. Combining rsync, bash script and cron, now I have backup system which does everything I need.
Features I wanted:
- Daily backup
- Only copy changed files
- Write a log with list of changed files
- Read a list of directories and synchronize them one by one
- Way to exclude some files or folders
So here’s the procedure.
1. Write a script
The script contains a list of directories that will be synced. Log is created in a subfolder, one file for every day. I’ve put the script in ‘/usr/local/bin/backup-all’ file.
#!/bin/bash
# backup procedure by Hugo Riley, www.gamesandcode.com
BACKUP_LOG=/usr/local/bin/backup-log/backup-log-$(date +%Y%m%d)
EXCLUDE_LIST=/usr/local/bin/backup-exclude
echo "************************" > $BACKUP_LOG
echo "System backup with RSync" >> $BACKUP_LOG
date >> $BACKUP_LOG
echo "************************" >> $BACKUP_LOG
FILES=(
'/boot /media/backup/filesystem'
'/home /media/backup/filesystem'
'/opt /media/backup/filesystem'
'/media/data/Dokumenti /media/backup'
'/media/data/Music /media/backup'
'/media/data/Projects /media/backup'
)
ELEMENTS=${#FILES[@]}
for (( i=0;i<$ELEMENTS;i++)); do
folderpair=${FILES[${i}]}
rsync --delete --delete-excluded -av $folderpair --exclude-from=$EXCLUDE_LIST >> $BACKUP_LOG
done
#end of script
Script must be executable to work so it needs to be chmoded:
chmod +x backup-all
Files and folders can be excluded from backup. Just add some filters to backup-exclude file:
.trash
Trash
Cache
.thumbnails
2. Make a schedule
To enable daily backup, you need to add it to the Anacron. It can be modified in ‘/etc/anacrontab’. I wanted my backup to start every day, ten minutes after Linux is finished booting (login is not necessary). After the editing, my anacrontab file looks like this:
# /etc/anacrontab: configuration file for anacron # See anacron(8) and anacrontab(5) for details. SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # These replace cron's entries 1 5 cron.daily nice run-parts --report /etc/cron.daily 7 7 cron.weekly nice run-parts --report /etc/cron.weekly 1 10 SystemBackup /usr/local/bin/backup-all @monthly 15 cron.monthly nice run-parts --report /etc/cron.monthly
4 Responses to “Linux system backup with RSync”
May 30th, 2008 at 2:41
I have a question…
with the line in /etc/anacrontab:
1 10 SystemBackup /usr/local/bin/backup-all
Does this mean the task is scheduled to take place daily, 10 minutes after the computer is first booted up?
May 31st, 2008 at 15:36
Yes, Andrew, you are correct. I didn’t make this clear enough, thanks for pointing this to me. I’ve edited the article so it’s clear now.
June 5th, 2009 at 13:30
Have you checked the option –files-from ? It seems to me that using it with a file like “to_backup” (see below) as well as rsync’s logging features you could eliminate all your bash code and just leave the rsync invocation. Here my rsync command with some explanations.
rsync -rihavz –files-from=to_backup –log-file rsync-log
–rsync-path=”rsync –log-file=/home/me/rsync-log” –delete-after
/home/me remote_machine:~
what this does:
-a is preserving permissions, and normally recursing (-r) when there
is no –from-file. As we read our backup directories from the to_backup file, we also need
-r recurse
-h human file sizes (13.4M instead of so and so many bytes)
-v verbose
-z compress
-i itemized list of stuff being transferred
–log-file specifies a log file name locally
–rsync-path path of remote rsync executable. We use this command in
order to tell the remote rsync to also log the transfers (in case
something fails in the client, the server knows which was the last
transfer).
–files-from specifies that the directories and concrete files to be
backed up are to be read from to_backup
–delete-after delete all files in server that have been
moved/deleted in client, but do so after the transfer.
remote_machine:~ dump the files on my home directory at the remote machine which is accessible via ssh with passphrase (if you don’t
know what this means, check this out:
http://sial.org/howto/openssh/publickey-auth/)
the to_backup file simply contains a list of
directories or single files to be recursively taken into account. If
you wish to have md4 hashing for identification of changed files
instead of timestamp and size or particular sets of excluded files,
check the rsync man page, which is quite readable.
Cheers,
Álv.
June 5th, 2009 at 18:50
Thanks, I wasn’t aware of “-files-from” option. It looks like it could get the job done. (I suppose it’s quicker to write the bash script than to read the whole man page…
)