It is a terrible thing to realize that you are stuck in a rut. Being in a rut effectively means that you’ve stopped advancing and life has evolved to monotony. No one likes to be around people in ruts, but it’s even worse to discover that you are personally trapped in one. And, most unfortunately, I am in a rut.
Don’t believe me? Take a look at the home page of this blog. You will likely notice that a full six of the ten most recent posts have dealt with one subject: backing up your computer. That’s pretty conclusive evidence of a rut.
Now, backing up your computer is a very important thing to do; you should do it regularly and have a plan. But … well … it’s boring. Talking, thinking and writing about nothing but backup is dull. As one of the doctors I work with likes to say, “That isn’t sexy. If I’m going to spend any time with it – women, food, wine; it doesn’t matter – it should be sexy.”
He’s got a valid point, backup is not “sexy” and I’d like to write about things that are, at least for a while. This, therefore, will be my last post on backups, archives, or servers for the relatively foreseeable future (technology is just too cool to lay it aside for too long). But before doing that, I want summarize where I ended up in my quest for the ultimate backup system.
Backup on Mac is taken care of, I use Time Machine to a Samba share. More adventurous persons than I might even say that this arrangement approaches sexy. It’s convenient, fast, and robust. It even covers disaster recovery.
Backup on Windows is also covered. The built-in file backup is easy to use and works well. Moreover, setting up a disaster recovery system is relatively painless.
But the third major operating system, Linux, is a bit of the odd-man out. Certainly, you can find some excellent backup systems, Back In Time is one such example. With a bit of work, you can even tweak it so that it is almost perfect. But it’s the “almost perfect” and closely related cousins (“mostly useful” and “good enough”) that are the problem. They have those stupid qualifiers – almost, mostly, enough – bolted on.
Any time you hear a qualifier, you can rest assured that you aren’t going to like what follows. Consider the rather innocuous phrase, “that may be a problem.” Here, the term “may,” makes an already bad situation much worse. Instead of specifying some probability of problemhood, it all but guarantees it. Positive qualifiers are just as bad.
As a result, it angers me that nearly every backup program available for Linux requires some kind of qualifier. It shouldn't be like this. Linux is a brilliant operating system in practically every way. It is highly integrated, wonderfully modular and tremendously easy to extend. So … after finding that nearly every backup utility in existence has failed to meet my needs, I found the situation intolerable and decided to do something about it.
I wrote my own.
Backgrounds and Backends
Fixation and ruts can make you do silly things like that. Instead of just accepting the limitations of an existing situation, a fixated person will demand that the world bend to their expectations. While this sometimes leads to great advances, more often it results in interpersonal disasters of epic scale. And the general rule is, the more menial the detail, the larger the scandal. (At this point, it might be argued that backup to an external hard drive versus across the network is a rather menial detail. To all such detractors, all I can say is: stuff it. This is my story.)
Luckily, however, the story of my backup utility isn’t nearly so sordid. This is probably because my needs are actually pretty simple. I need a backup utility to do just a few things, but I require that it does them well:
- It should create versioned snapshots of my drive. This lets me restore a file to any number of past states.
- The utility should only transfer the portions of the file or directory that have changed. This makes the backup operation quick and efficient.
- I should be able to backup over a network or across the internet. (Very important!)
- Restoring a file should be quick, easy and painless. A backup is only as good as your ability to get at the information. And like it or not, information will be lost due to disaster, carelessness or miscellaneous stupidity.
Being a somewhat clever person, I didn't to avoid duplicating as much work as possible. As a result, I started creating my program by reviewing the application that comes closest to doing what I want: Back In Time. The Back In Time user interface is simple and elegant. More importantly, though, the configuration options make sense. Whether you want to run a backup, give your snapshot a name, or just get rid of it; you don’t have to go hunting to find the controls. Additionally, Back In Time makes good use of existing open source programs to actually do the backing up. Though they might be a bit old, rsync and the Unix copy command, cp, are a formidable duo.
But I quickly discovered something unfortunate, Back In Time really couldn’t be tweaked to meet all of my requirements. You see, it utilizes a technique called hard-linking to create space efficient repositories. But hard links only work as long as they are on the same hard drive. Because I want to store my files on the network or across the internet, that isn't good enough.
So, I started to look elsewhere for other options, which is when a colleague pointed me toward a command line script called Duplicity. In a striking bit of irony, while the GUI tools of Linux backup are all fundamentally flawed, the command line versions are the best in existence. They easily accomplish all of the requirements on my list, and do a great many things I’ve never even heard of. As a result, if you are willing to roll up your sleeves and hit the learning curve, there is nothing that you can’t accomplish. And, as you might deduce from my glowing introduction, Duplicity makes short work of my requirements list.
Can it do incremental backup? Yes. What about backup over the network? Ditto. In fact, it handles most protocols I’ve heard of, and quite a few that I haven’t. Should incremental through ssh, webdav, ftp, smb, or imap be insufficient, it also supports backup to Amazon S3. File restoration is also relatively painless, though you have to work through the command line interface to do so.
And that’s only for starters, Duplicity also supports encryption via passphrase or passkey, compression of the files into archives, and local caching of file signatures so that backup operations are lightning quick. I decided relatively quickly that Duplicity would be the ideal backend for my utility.
Frontend and Features
But while a solid foundation is a great starting point, it can only get you so far. After all, you don’t brag up the brickwork and reinforced concrete of your newly purchased home, you’re far more likely to show off the kitchen and home theater.
For a program, the points of interest are going to be the user interface. Which makes two lessons learned while working on LyX-Outline very important: 1) It’s really hard to put together an interface that is both intuitive and uncluttered. 2) I’m really bad at it. Both points prompted me to act on the advice of a famous painter:
Good artists copy; great artists steal.
Thus, I decided to steal the user interface from Back In Time. (It’s open source, which makes it okay.) And speaking from a practical standpoint, why wouldn’t I? The developer spent a great deal of time working out how he wanted his program to work. It would be silly to duplicate that effort. Moreover, the fact that I was able to rave about it for nearly 2800 words means that it wasn't wasted effort. As you browse the menus and configuration panes of my derivative tool, you will probably notice that it bears a striking resemblance to other programs. A few things will be “innovatively different,” but not many. Just remember, I stole the interface. Wholesale. But I’m a discriminating thief, and only stole the best parts.
Which brings us to the real point of interest of this post and my program: the feature list. In part 2 of this article, we’ll take a look at my little creation, which I have unimaginatively dubbed “Time Drive.”