I backup regularly, 4 times daily with the fantastic utility rsnapshot, and between runs I keep the backup disk safe by remounting in read-only mode. How I remount programmatically will be the subject of another post, but basically the backup disk has to be in read-write mode during the backup, but otherwise I keep the disk in read-only mode so none of my backups get accidentally overwritten.

A downside is that after a while, mount starts complaining about the disk being re-mounted too many time between checking the filesystem with e2fsck. It would be easy to set the time between mount checks to a very high number using the utility tune2fs (tune2fs -c max-mount-counts) but eventually I would run into this same problem. Besides, there is probably some logic behind having the default max-mount-count be set to something low, like the default value of 39.

So, the magic of bash scripting and cron jobs can help out.

First, the script needs to figure out where my backup disk is mounted. In my /etc/fstab file, I use a mount point in my system of /.snapshots. Here’s the corresponding line in /etc/fstab:

UUID=0dc5c715-96aa-4299-aec4-0a226da191c9 /.snapshots             ext3    ro,noauto,noexec        0 0

So, if I run /bin/mount from the command line, I can figure out the device name of my backup disk:

# /bin/mount |grep .snapshots
/dev/sdc1 on /.snapshots type ext3 (ro,noexec)

See the final script below for teasing out the device name from this line using sed.

Next, dumpe2fs gives us lots of information about a device’s filesystem, including how many times it has been mounted (mount count) as well as the “max_mount_counts” set by tune2fs (or set by default). See the script below for details.

Now, the script needs to get the work done by running e2fsck, but we need to be careful. e2fsck should only be run on UNmounted devices. If we can’t unmount the device, we shouldn’t run e2fsck. Also, I don’t want to run this script if we’re int he middle of a backup session. Fortunately rsnapshot let’s us know if that’s the case by writing the pid to a file, /var/run/rsnapshot.pid, while it’s running. We can also take advantage of this by writing our own rsnapshot.pid file so that we don’t start a backup session if e2fsck is running.

Finally, I want this to be run from cron so I don’t have to worry about this all the time. Unfortunately (or perhaps fortunately), e2fsck won’t run without an interactive session (from the command line). It won’t run from a script. So, I need to run my cron script in such a way that it will report to me (through email) that e2fsck needs to be run from the command line. The script below therefore accepts a command-line option (-reportonly) that does this. My /etc/cron.d/backup-server therefore looks something like this:

MAILTO=root

12 */4 * * * root /bin/nice /usr/bin/rsnapshot -v hourly
40 0 * * * root /usr/local/bin/fsck_backup_disk -reportonly

This sets my backup routine to run 4 times daily at 12 after the hour, and the script below (which I named fsck_backup_disk) runs daily at 40 minutes after midnight. I then get notified by cron through email if I need to go to my command line and run fsck_backup_disk manually.

Here’s my full /usr/local/bin/fsck_backup_disk:

#! /bin/bash

backupDisk=`/bin/mount |/bin/grep snapshot |/bin/sed ‘s/\(\/dev\/sd[a-h][1-3]\).*/\1/’`

mountCount=`/sbin/dumpe2fs -h $backupDisk 2> /dev/null |/bin/grep “Mount count” |/bin/sed ‘s/^.* \([0-9]*\)/\1/’`

maxMountCount=`/sbin/dumpe2fs -h $backupDisk 2> /dev/null |/bin/grep “Maximum mount count” |/bin/sed ‘s/^.* \([0-9]*\)/\1/’`

# If mountCount > maxMountCount then will need to run e2fsck
# but not on a mounted system! Will need to umount, but make sure it’s not in use first

# If this script run from cron, then pass the option “-reportonly” since e2fsck won’t run without the command line.

if [ -z $1 ]; then
        /bin/echo -n “The backup disk ($backupDisk) has been mounted $mountCount out of maximum $maxMountCount times. “;
        if [ $mountCount -gt $maxMountCount ]; then {
                # check whether backup is running.
                if [ ! -f “/var/run/rsnapshot.pid” ]; then
                        /bin/echo -e “Checking backup disk file system.\n”;
                        /bin/touch /var/run/rsnapshot.pid;
                        /bin/umount $backupDisk && /sbin/e2fsck $backupDisk
                        /bin/mount $backupDisk;
                        /bin/rm -f /var/run/rsnapshot.pid;
                fi
        } else {
                /bin/echo “No need to check backup disk file system.”;
        } fi
elif [ $1 == ‘-reportonly’ ]; then
        if [ $mountCount -gt $maxMountCount ]; then
                /bin/echo -n “The backup disk ($backupDisk) has been mounted $mountCount out of maxiumum $maxMountCount times. ”
                /bin/echo “Please run /usr/local/bin/fsck_backup_disk.”;
        fi
fi