First of all I feel bad, because I promised John Goerzen to share my experiences,
but as with other things I am lagging behind.
Still better late than never.
A few words on rsnapshot
rsnapshot is a great tool for making efficient backups. It basically creates directories like
daily.0
daily.1
daily.2
daily.3
...
daily.6
The directory contents are hard linked on each other and then it is overwritten when the contents (files) are changed.
The way this is achived is by cp -al daily.0 daily.1 then rsync-ing the fresh content to
the pre-populated directories. Sometimes rm -rf daily.N (where N is the number of days you want to keep)
is run remove old content. (For simplicity I do not talk about weekly and monthly features of rsnapshot).
There is also a rollback mechanism in place if the backup fail to get the last consistent state for the next run as a base.
The problem is that cp -al and rm -rf and the rollback mechanism itself is an expensive operation
and does not use the possibilities of ZFS.
ZFS snapshots
ZFS have a convenient way to create snapshots of datasets which are consistent point-in-time state of the filesystem. The snapshots are read-only, mountable and almost free with zfs. As ZFS is a copy-on-write filesystem when you modify a file it is copied first and when you have a snapshot the original is kept after it is changed. Only the changed content takes space.
ZFS and rsnapshot
I choose an easy and convenient way to integrate the ZFS snapshotting from rsnapshot some may call it a hack.
There is a cmd_cp configuration parameter and I wrote the script that does the job for me:
#!/usr/local/bin/bash
#echo "snapshot created at $(date)" > $2/info
# name of the volume
zname=$(dirname $(echo $2 | cut -b 2-))
# name of the future snapshot
bsname="$zname@rsnap-$(date +%F)"
sname=$bsname
for i in 1 2 3 4 5
do
# exit if success
zfs snapshot $sname
if [ "x$?" == "x0" ]; then
echo "backup started at $(date)" > $2/info
exit 0
fi
# iterate over it if already exists
sname=$bsname-v$i
done
echo ERROR zfs_cp: something is really broken
exit 1
So every rsnapshot directory is a seperate dataset and when the rsnapshot starts the tedius work and of running cp
is eliminated. I will always only have daily.0 there never will be a daily.1 as my script never creates it.
This way the rm -rf and the rollback part is also eliminated as there is nothing to delete and nothing to rollback to.
If a backup does not complete for one reason or another the next one supposed to correct it.
One thing that is missing: Eliminating and removing of old snapshots. I to keep the 7 recent backup then keeping 4 weekly is fine and 6 monthly. There are tools available to do that I need to look at that.
zfs list -t all sniplet:
NAME USED AVAIL REFER MOUNTPOINT
...
backup14q1/rsnapshot_1 1.85T 15.6T 1.45T /backup14q1/rsnapshot_1
backup14q1/rsnapshot_1@rsnap-2014-01-08 5.38G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-09 3.82G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-10 3.82G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-11 4.06G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-12 3.86G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-13 5.72G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-14 3.89G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-15 3.90G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-15-v1 3.66G - 1.44T -
backup14q1/rsnapshot_1@rsnap-2014-01-17 3.89G - 1.45T -
...
backup14q1/rsnapshot_1@rsnap-2014-02-12 4.21G - 1.45T -
backup14q1/rsnapshot_1@rsnap-2014-02-13 4.19G - 1.45T -
Summary
Some features of rsnapshot like:
- weekly, monthly snapshots
- rollback in case of failures
are lost in the process, but overall I am satisfied with the end results. Further improvements are possible.