Installing Duplicity on Mac OS X

 | October 7, 2009 3:20 pm

While Mac OS X shares many things in common with other Unix operating systems, it also has a couple of missing parts.  This can make installing software more difficult when compared to Linux or other *nix variants, and unfortunately, Duplicity happens to be one of those cases.

But just because it is more difficult to install Duplicity on Mac doesn’t mean that it isn’t worthwhile.  Duplicity is one of the best command line backup programs available anywhere.  Using the same program, you can backup to a local hard drive, FTP server, over SSH or even to Amazon S3.  It uses the rsync algorithm, which means that backups happen quickly and only the parts of files that actually changed get copied.  Subsequently, backups are smaller.

Nor does it mean that installing Duplicity is actually all that hard, it’s just a bit tedious.  Like many other *nix programs, Duplicity requires a number of additional programs (called dependencies) that you will have to search out and install separately.  To make that process easier, this article explains how.  Below, you’ll find links to the download pages and instructions for how to compile each of Duplicity’s dependencies on Mac OS X.  The instruction set has been tested both on Mac OS X 10.5 (Leopard) and Mac OS X 10.6 (Snow Leopard).

XCode XCode

The first dependency you will need is a rather large one.  Several of the software packages need to be compiled from source.  But unfortunately, Mac OS X doesn’t install the compilers by default.  Thus, you will either need to install them from the Mac OS X installation CD, or by getting the XCode tools from the Apple Developer Connection (ADC)Note: ADC requires that you have an Apple ID – which is free – to log in into the site.

After you log-in, download the XCode developer tools package.  Once XCode has finished downloading, run the package installer.  It will ask you to select the destination drive and what components of the software you would like to install.

Select the installation disc where you want to install XCode the developer tools

Choose the Develoepr Tools Essentials, System Tools and Unix Development Support options.  This will automatically install and configure the GNU tool chain and the GCC compiler collection.  Once it has finished, you can access the tools by going to the “/Developer/” folder on the root of the drive.

Choose the components you would like to install

After XCode is finished installing you are ready to build and install the other necessary dependencies.

Note: All other steps in this tutorial will be run from the command line terminal.

LockandKey5

GnuPG

GnuPG handles the encryption and decryption of tar archive files.  If it isn’t installed and configured correctly, Duplicity (and by extension) Time Drive simply won’t work.  On Linux and other Unix operating systems, GnuPG is typically included as part of the default installation.  This is not true on Mac OS X, however.  As a result, you will need to download, compile and install it from source.

You can find the source files on the project download page or the associated ftp site.  Duplicity will work with version GnuPG version 1.4 or 2.0.  The biggest difference between the two is that version 2.0 is slightly more modular, and as a result, has a speed advantage.  From a practical standpoint, however, unless you are running large, frequent backups, you are not likely to notice any difference.  I have been able to compile both without any difficulty on both Leopard and Snow Leopard, so the choice is largely one of preference.

After downloading the most recent version, extract the archive by typing:

tar –xf gnupg-x.x.x.tar.gz2

Where x.x.x is the version of GnuPG that you downloaded (at the time of this writing, it is version 2.0.12).  This will create a new directory called gnupg-2.0.12.  From that directory, you can configure, compile, test and install:

cd gnupg-2.0.12
./configure
make; sudo make install

When finished, return to the working directory by typing:

cd ..

GnuPG Interface – Python Module

Now that you have built and installed GnuPG, you will next need to install the Python bindings so that duplicity can make use it.  You can get these from the GnuPG Interface Module project page, specifically from the download section.  Grab the most recent version (0.3.2).  To install, extract the folder, enter the setup directory and then use python to run the “setup.py” script:

tar -xf GnuPGInterface-0.3.2.tar.gz
cd GnuPGInterface-0.3.2
sudo python setup.py install

NcFTP

NetworkArchive5NcFTP enables Duplicity to make backups to FTP servers.  The download files can be found on NcFTP’s download page, which lists both the NcFTPd Server and the NcFTP client.  We want the most recent version of the client source code (version 3.2.3).

Like GnuPG, we need to configure, compile and install:

tar –xf ncftp-3.2.3-src.tar.bz2
cd ncftp-3.2.3
./configure
make; sudo make install

Boto – Amazon S3 Support

To use Amazon as a backup destination, you need to grab the  Boto python module.  Boto provides a structured and integrated interface to most of the features offered by Amazon Web services, most importantly: the Simple Storage Service (S3).  You can get the most recent version (1.8d) from the Boto downloads page on Google Code.

From there, the installation procedure is similar to that for the GnuPG python interface module.  Extract the archive, enter the program folder and use python to run the setup script.

tar -xf boto-1.8d.tar.gz
cd boto-1.8d
sudo python setup.py install

Duplicity

Once you have installed GnuPG, the GnuPG python interface, NcFTP and Boto, you are ready to install Duplicity.  Because it is a python program, the steps are identical to those of Boto and the GnuPG python interface.  Download the source code from the project home page (keeping in mind that Time Drive requires at least version 0.6.04); then, extract it the same way that you did the previous libraries and enter the source code directory:

tar –xf duplicity-0.6.04.tar.gz
cd duplicity-0.6.04

Finally, run the python setup script:

sudo python setup.py install

Duplicity will byte compile and be copied to the python packages directory.

When finished, you can test the installatin by typing:

duplicity –help

If successful, you will see the text from the duplicity manual page.


Similar Posts:

6 Responses to “Installing Duplicity on Mac OS X”

Henrik Olsen wrote a comment on March 14, 2010

Running .configure on gnupg-2.0.15 on Mac OS X 10.6.2 gives me this ending:

configure:
***
*** You need libgpg-error to build this program.
** This library is for example available at
*** ftp://ftp.gnupg.org/gcrypt/libgpg-error
*** (at least version 1.4 is required.)
***
configure:
***
*** You need libgcrypt to build this program.
** This library is for example available at
*** ftp://ftp.gnupg.org/gcrypt/libgcrypt/
*** (at least version 1.4.0 using API 1 is required.)
***
configure:
***
*** You need libassuan to build this program.
*** This library is for example available at
*** ftp://ftp.gnupg.org/gcrypt/libassuan/
*** (at least version 2.0.0 (API 2) is required).
***
configure:
***
*** You need libksba to build this program.
*** This library is for example available at
*** ftp://ftp.gnupg.org/gcrypt/libksba/
*** (at least version 1.0.2 using API 1 is required).
***
configure:
***
*** It is now required to build with support for the
*** GNU Portable Threads Library (Pth). Please install this
*** library first. The library is for example available at
*** ftp://ftp.gnu.org/gnu/pth/
*** On a Debian GNU/Linux system you can install it using
*** apt-get install libpth-dev
*** To build GnuPG for Windows you need to use the W32PTH
*** package; available at:
*** ftp://ftp.g10code.com/g10code/w32pth/
***
configure: error:
***
*** Required libraries not found. Please consult the above messages
*** and install them before running configure again.
***

Apparently a lot of libraries are missing.

Henrik Olsen wrote a comment on March 14, 2010

All above libraries missing above have been manually downloaded and installed now.

By the way, ncftp won't compile on a clean Snow Leopard install, since the configure script refers to the 10.4 SDK which is not in Xcode in OS X 10.6. Correct as needed. I have informed ncftp maintainers about this now.

Rob Oakes wrote a comment on March 14, 2010

Thanks for the head up. I have had a lot of trouble compiling software on Snow Leopard. It's been a while since I tried to compile NCFTP. I have been having serious problems with PyQt and have not been using Time Drive on Mac for a while. It's probably time that re-visit this topic and update the instructions.

Henrik Olsen wrote a comment on March 15, 2010

Using

./configure CC="gcc -arch i386"

for all dependencies (as needed for gnugpg according to http://macgpg.sourceforge.net/), and having used

CFLAGS="-arch i386 "
python setup.py --librsync-dir=/usr/local build
python setup.py install --prefix=/usr/local
export PYTHONPATH='/usr/local/lib/python2.6/site-packages/'

for the final duplicity install (to get it in /usr/local), I got an apparently clean install.

But... Running a simple version check gives this :(

duplicity -V
Traceback (most recent call last):
File "/usr/local/bin/duplicity", line 41, in
from duplicity import collections
File "/usr/local/lib/python2.6/site-packages/duplicity/collections.py", line 29, in
from duplicity import path
File "/usr/local/lib/python2.6/site-packages/duplicity/path.py", line 36, in
from duplicity import librsync
File "/usr/local/lib/python2.6/site-packages/duplicity/librsync.py", line 29, in
import _librsync
ImportError: dlopen(/usr/local/lib/python2.6/site-packages/duplicity/_librsync.so, 2): Symbol not found: _rs_file_copy_cb
Referenced from: /usr/local/lib/python2.6/site-packages/duplicity/_librsync.so
Expected in: flat namespace
in /usr/local/lib/python2.6/site-packages/duplicity/_librsync.so

Googling _rs_file_copy_cb only finds one entry, suggesting using Python 2.6 instead of Python 2.5 (already using 2.6). Any ideas/similar experience?

Henrik Olsen wrote a comment on March 15, 2010

Might be that Python 2.6 is only 64-bit on OS X 10.6. Hmmmm, then what to do...? When gnugpg apparently needs to run in 32-bit.

Rob Oakes wrote a comment on March 15, 2010

The version of python that ships with Snow Leopard is a very strange duck. Apparently, it can execute both 32 bit and 64 bit code, but getting the code to compile can be a bit difficult. This is the problem that I've run into with PyQt. Installing the pre-built binaries (from the Qt website) doesn't work (or at least it doesn't work well). If you build Qt yourself for x86_64, then you can get PyQt to build and install, but will get errors at run-time. I have not yet found aa work-around, or the magical set of compiler commands and luck which will yield success.

If anyone else has ideas or workarounds, I'd love to hear them.