• HOME
• CONSULTANCY
• INSTALLATION
• SUPPORT
• HOSTED SERVICES
• RESCUE CENTRE
• TESTIMONIALS
• CONTACT

Lucid Nonsense

rsync Binary Installer

rsync is a fantastic application, very flexible, very scriptable, and very efficient. We use it as a backup tool for a number of our customers (see our hosted services page), mainly for backing up their servers to our storage as a disaster recovery measure. It can be used between both remote servers and locally between folders or hard disks. It can even do a good impression of Time Machine style backups (with a bit of work) in situations where you can’t use Time Machine itself. It’s also both free and open source, which is always nice.

There are, however, a number of things to look our for with rsync. First off it’s a command line application, so you need to get your hands dirty in the Terminal to run it. It’s also a tool that can really benefit from learning some of the more advanced features. Secondly it doesn’t include any scheduling functionality itself, so with Leopard or Tiger for instance you would need to set up a launchd job to manage the timing. With the power and flexibility of launchd though that’s not necessarily a bad thing, as an example using launchd’s WatchPaths feature you could start rsync whenever a particular volume was mounted. Thirdly, and more importantly, Apple ship Leopard with a really out of date version of rsync: version 2.6.9. This version will work, but it will miss some file attributes and not give you a perfect backup. For a complete backup you really need rsync 3.0.5 with the filediffs and crtimes patches applied. If you want to handle that yourself, or you want to use a newer version (3.0.6pre1 is in testing as of writing) then Mike Bombich has some fantastic instructions for building rsync, while including all necessary patches. That page at bombich.com also has some good instructions for setting up public/private keys and authorized_keys files to allow secure logins without a password which is the best method for automating secure remote rsync backups.

Sometimes though, you may want to quickly install rsync without installing Apple’s Xcode tools. I needed to do that the other day on a server but didn’t have a Mac with the dev tools handy. Strangely I couldn’t find a binary install for rsync 3.0.5 so I’ve put up a binary installer for a fully-patched version of rsync 3.0.5. This installer will work for Intel Macs running Leopard, and will install rsync in /usr/local/bin/, so won’t replace your existing version of rsync. To run this new version you need to specify the full path of /usr/local/bin/rsync. You can grab the installer from the downloads section.

There are lots of examples of rsync commands out there, and lots of sites that cover it in a lot of detail but I’ve added a couple of useful Mac-friendly ones below that work with 3.0.5.

First off, this command will fully sync a drive or folder to a remote server (all the commands on this page are single commands that should be entered on one line, they have been wrapped to fit the page):

/usr/local/bin/rsync -aNHAXxs --fileflags --force-change --delete
--rsync-path="/usr/local/bin/rsync" '/Volumes/Studio/'
root@server.example.com:'/Volumes/BackupFiles/Studio/'

That command will synchronise the Studio volume on the local Mac to the Studio folder on the BackupFiles volume on server.example.com. Files not on the source volume would be deleted, and only files that are different on the destination would be transmitted. It’s possible to compress files as they’re being transmitted by adding the -z switch, although due to the extra work that has to be done to compress and decompress the data, and the type of files you’re sending, compression may actually slow transfer speeds even if it decreases the total amount of data being sent. It’s worth testing with your connection and data to see what works best.

If you’re using rsync over a network or the internet, and don’t want to saturate the connection, the bandwidth can be limited by adding the --bwlimit= flag in there, with the limit specified in KB/s (kilobytes per second), for instance to limit the above command to using 2Mb/s (megabits per second), you would add --bwlimit=256.

Local backups are formulated in the same way as network based backups with the “from” and “to” paths as before, but omitting the “username@host:” section, so this would backup your Pictures folder to an external hard drive called Backup:

/usr/local/bin/rsync -aNHAXxs --fileflags --force-change 
--rsync-path="/usr/local/bin/rsync" '/Users/user/Pictures/' '/Volumes/Backup/'

That command doesn’t include the --delete flag so will not remove files that are no longer in the source folder. It would overwrite files that have been changed though.

Finally, this command uses the --backup and --backup-dir switches to keep files that rsync would otherwise delete in a separate folder. You can create a fairly simple shell script based on this that will create folders with the specified date, and then keep the main backup in sync, while keeping daily folders full of files that were changed or deleted on the source.

/usr/local/bin/rsync -aNHAXxs --fileflags --force-change --delete --backup 
--backup-dir="/Volumes/Backup/18-4-09/" --rsync-path="/usr/local/bin/rsync"
'/Users/username/Pictures/' '/Volumes/Backup/Pictures/'

There are lots more options and methods with rsync but those are some of the main options to get started with. The important bits for getting reliable backups on a Mac are the -aNHAXxs --fileflags --force-change switches, after that you can start tweaking to make things work the way you need. It’s worth reading the documentation though because it is a very powerful tool and can do a lot more than is at first apparent.