How to install python subversion bindings needed by hg convert on Mac OS X 10.8?

macos, mercurial, python, svn

Solution

Using homebrew

The easiest way to get this working is to install `subversion` from source using `homebrew` and the `--with-python` flag.

First, make sure you have your command-line tools installed. With Mavericks and Xcode 5, most commands like `cc` just work even though the command-line tools aren’t fully installed. If you don’t have a `/usr/include` directory, then first you need to run

$ xcode-select --install

Once your command-line tools are installed, run:

$ brew install subversion --with-python
$ mkdir -p ~/Library/Python/2.7/lib/python/site-packages
$ echo $(brew --cellar)/subversion/1.8.5/lib/svn-python \
    > ~/Library/Python/2.7/lib/python/site-packages/svn.pth

You can test the bindings by running the unit tests:

$ svn co http://svn.apache.org/repos/asf/subversion/tags/1.8.5/subversion/bindings/swig/python/tests
$ cd tests && python run_all.py

Using Apple’s source code

This works for Mountain Lion, but needs tweaks for Mavericks and results in failing unit tests; see Simon Wright’s answer to this question.

It is possible to build the subversion bindings for Python using Apple’s version of the subversion source code. The resulting module will be exactly compatible and link against all the system libraries. And then `hg convert` will just work.

Here’s how to do it:

Download the subversion tarball from opensource.apple.com

Unpack it and configure it:

cd subversion-52/subversion && ./configure

In `subversion/bindings/swig/python`, add this Makefile, being sure to change leading whitespace to tabs:

SHELL = /bin/bash -eu

CC = gcc -g -O2
CFLAGS = -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK \
    -I ../proxy \
    -I ../../.. \
    -I ../../../include \
    -I /usr/include/apr-1 \
    -I libsvn_swig_py \
    -I /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 \

LDFLAGS = \
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib \
    /usr/lib/libsvn_*-1.0.dylib \
    /usr/lib/libapr-1.dylib \

.PHONY: build egg test clean install

test: egg
    mkdir -p tmp && cd tmp \
      && PYTHONPATH=../egg python -S -m svn.tests.run_all

DESTDIR=$(HOME)/Library/Python/2.7/lib/python/site-packages
install: egg
        mkdir -p "${DESTDIR}"
    rm -rf "$(DESTDIR)/svn.egg"
    cp -R egg "$(DESTDIR)/svn.egg"
    echo './svn.egg' > "$(DESTDIR)/svn.pth"

egg: build
    rm -rf egg
    mkdir egg
    cp -R svn egg
    cp -R tests egg/svn
    touch egg/svn/tests/__init__.py
    mkdir egg/libsvn
    cp *.py egg/libsvn
    cp *.so *.dylib egg/libsvn
    # https://blogs.oracle.com/dipol/entry/dynamic_libraries_rpath_and_mac
    for F in egg/libsvn/*.so; do \
      install_name_tool -change libsvn_swig_py.dylib \
        '@loader_path/libsvn_swig_py.dylib' "$$F"; \
    done
    strip -x egg/libsvn/*.so
    touch egg/libsvn/__init__.py
    mkdir -p egg/EGG-INFO
    echo 'Version: 1.6.8' > egg/EGG-INFO/PKG-INFO

build: libsvn_swig_py.dylib _client.so _core.so _delta.so _diff.so _fs.so _ra.so _repos.so _wc.so

libsvn_swig_py.dylib:
    $(CC) $(CFLAGS) -shared -dynamic libsvn_swig_py/swigutil_py.c -o $@ $(LDFLAGS)

_client.so: svn_client.c
_core.so: core.c
_delta.so: svn_delta.c
_diff.so: svn_diff.c
_fs.so: svn_fs.c
_repos.so: svn_repos.c
_wc.so: svn_wc.c

_%.so: svn_%.c libsvn_swig_py.dylib
    $(CC) $(CFLAGS) -bundle $^ -o $@ $(LDFLAGS)
_%.so: %.c libsvn_swig_py.dylib
    $(CC) $(CFLAGS) -bundle $^ -o $@ $(LDFLAGS)

clean:
    rm -rf *.o *.so *.dylib *.dSYM

Then run `make` in that directory.

If the unit tests passed in the last step, you now have fully-functional Python bindings for subversion! Run `make install` if you’re happy, and `hg convert` will start working.

As is so often the case, it was easier to rewrite the build system from scratch than to figure out the existing one.

All the scripts from Apple and Subversion really complicate it, but all you really need to do is copy the `.py` files, build a shared library with the common code, and then build each C-language Python module. Figuring out which include files and libraries to use is straightforward: try to build it starting without any includes or libraries, and when the build fails, add a reference to the missing include path or library that is causing the build failure. The tricky part is telling the C-language modules like `_core.so` where to find the dynamic library of common code.

Problem

Original source