Automated Backup w/ rsync over SSH

There are four main sections to getting this up and running

  1. Introduction
  2. Installing/Configuring Required Software
  • OpenSSH
  • rsync
  • smbfs (optional)
  • Auto-mounting a Windows share (optional)
  • Creating a script to do the work for you
  • Automating the backup using cron jobs
  • Introduction

    Before going over everything here, I'd like to point out why this information is useful and define a few terms. Using rsync to create a backup server solution is entirely free and conserves bandwidth. The reason it conserves bandwidth is rsync itself only sends the parts of each file that have changed. FTP sends the entire file each time even if only one word was changed in a file. Sending all of this traffic over SSH ensures security and also protects you from having to run a full rsync server. Key Terms:

    • Backup server: the PC that will be issuing all the rsync commands and storing the backups
    • Host computer: the PC where we will be getting the files from

    With all of that out of the way, it's just a matter of setting everything up.

    Installing/Configuring Required Software

    In order to get things going you'll need to install a few pieces of software. In my situation I had a Windwos share that I wanted to back my home folder up to. In order to play around with Windows shares you'll need to also install the samba client.

    OpenSSH

    Simple to install with one line:
    sudo apt-get install openssh-server

    Once this is done you will be able to connect but it requires a password entry. This is sort of a problem when we're looking at automating the entire backup process so in order to solve that problem we'll use ssh keys instead. From your backup server we'll need to create an ssh key by issuing the following:

    ssh-keygen -t dsa

    Now you can either copy the key manually in some fashion or do it over an SSH connection with this:

    scp ~/.ssh/id_dsa.pub username@x.x.x.x:./id_dsa.pub

    Note that username is the local account on the host and x.x.x.x is the IP of the host. What this will do is copy the public key from the server to our host. After that we need to tell our host PC to use that key as authorization instead of asking for a password.

    cd .ssh
    touch authorized_keys2
    chmod 600 authorized_keys2
    cat ../id_dsa.pub >> authorized_keys2
    rm ../id_dsa.pub
    

    Here you can see we just change directory to .ssh and use touch to create the file authorized_keys2. Once the file is created lock it down and only allow the current user to change the file and finally add the public key then remove the original file containing the key. When you finish this try it out. If you're able to connect to your host pc from the backup server over SSH with no password everything has been setup correctly.

    rsync & smbfs

    These two are simple to install with just one line:

    sudo apt-get install rsync 
    sudo apt-get install smbfs 
    

    Keep in mind that smbfs is not required it just depends on your setup. If you would like to backup to/from a Windows share then you will want this along with the next section. Otherwise you can skip down to automating the backup using cron jobs.

    Auto-mounting a Windows share (optional)

    To mount a windows share everytime you boot up you will need to modify your /etc/fstab file. You only need to add one line at the end of this file like this:

    //x.x.x.x/Backups /media/Backup cifs iocharset=utf8,credentials=/root/.smbcredentials,uid=1000 0 0

    Of course you must replace x.x.x.x with the IP of your Windows computer and "Backups" with the share name. You will also need to create the file /root/.smbcredentials.

    .smbcredentials

    username=MyUsername@domain.com
    password=MyPassword
    

    This is a security risk because anyone with root priviledges can read your username/password in plain text so we must lock this down as much as possible.

    sudo chown root /root/.smbcredentials
    sudo chmod 600 /root/.smbcredentials 
    

    Also as a side note, you won't need "@domain.com" if your PC is not part of a domain. Once all this is down you can mount everything by issuing this:

    sudo mount -a

    You should now be able to access the share by browsing /media/Backups.

    Creating a script to do the work for you
    We're all lazy. Especially with making sure we have up-to-date backups. So to make it easier I've compiled a simple script that will ping the host to make sure it's alive and then execute the rsync command.

    rsync_backup.sh

    #!/bin/bash
    awk '	function pinger(count,ip) {
    		command = "ping -c "count " " ip		
    		while (( command | getline res )> 0 ) {	
    			if ( res ~ /0 received|100% packet loss/ ) {
    				close(command)
    				return 100			 
    			}
    		}
    	close(command)
    	return 0
    	}	
    	BEGIN { 	
    	IP="x.x.x.x"	
    	if ( pinger(2,IP) == 0 ) {
    		system("rsync -raz --delete --rsh=ssh user@x.x.x.x:/home/user /media/Backup")
    	}
    }'
    

    This will of course need to be modified to work with your setup. Replace x.x.x.x with your hosts IP and /home/user with the directory you would like to backup.

    rsync -raz --delete --rsh=ssh user@x.x.x.x:/home/user /media/Backup

    Here you can modify the actual rsync command used. The options r, a, & z are used for recursive, archive, and compress the data. The delete option deletes files that no longer exist on the source (in this case /home/user) and rsh tells rsync to use ssh. Take a look at the man page for further information about available rsync commands.

    Automating the backup using cron jobs

    To run this script everyday at 11PM you can simply add a cron job.

    crontab -e

    The above starts your default text editor in the file that holds cron jobs. The next line is what should be added to run the script at a certain time everyday.

    0 23 * * * /path/to/rsync_backup.sh

    Take a look at the crontab man page for a more detailed explanation of how cronjobs work.

    Summary

    Once this is all setup it's easy to modify to backup multiple PCs from the single backup server. For Windows PCs you can create a share or use the administrative share (\\comp\c$\path\to\files) and auto-mounting in order to back up PCs without installing SSH or rsync. This concept also works for Linux environments but does require a full samba install and configuration. That's the beauty of rsync, you can really do whatever you would like with it. If you have any questions or feel stumped feel free to contact me.

    Back to my projects

    Tags: