cython - distutils vs cmake: linking against libpython?

cmake, cython, distutils, python

Solution

I realized the source of the issue was related to the `cython-cmake-example` and how its `UseCython.cmake` `cython_add_module()` function would explicitly link the library against libpython.

What I ended up doing for my own use, since I do not know if this is a completely portable solution, was to add a flag to that function to say `DYNAMIC_LOOKUP`:

function( cython_add_module _name _dynamic_lookup )
  set( pyx_module_sources "" )
  set( other_module_sources "" )
  foreach( _file ${ARGN} )
    if( ${_file} MATCHES ".*\\.py[x]?$" )
      list( APPEND pyx_module_sources ${_file} )
    else()
      list( APPEND other_module_sources ${_file} )
    endif()
  endforeach()
  compile_pyx( ${_name} generated_file ${pyx_module_sources} )
  include_directories( ${PYTHON_INCLUDE_DIRS} )
  python_add_module( ${_name} ${generated_file} ${other_module_sources} )
  ### Added here ##
  if( ${_dynamic_lookup} )
    message( STATUS "Not linking target ${_name} against libpython" )
    set_target_properties( ${_name} PROPERTIES LINK_FLAGS "-undefined dynamic_lookup")
  else()
    target_link_libraries( ${_name} ${PYTHON_LIBRARIES} )
  endif()
endfunction()

Now I can call `cython_add_module` and it won't link against libpython.

Problem

I had asked another question that was a bit too complicated for a straight answer so I boiled it down to this basic question... When I build my `aModule.so` using a standard cython distutils, it does not seem to be linked against `libpython`: ``` $ otool -L aModule.so aModule.so: /usr/local/lib/libboost_thread-mt.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/local/opt/thrift/lib/libthrift-0.9.0.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11) ``` But when I build with a cmake setup, it keeps producing a linker command that links `libpython` into the .so : ``` $ otool -L aModule.so aModule.so: /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1) /usr/local/opt/thrift/lib/libthrift-0.9.0.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/local/lib/libboost_thread-mt.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0) ``` The module produced by distutils seems to work fine with any of my python2.7 installation (the system, or a virtualenv for my project). Whereas the cmake crashes with a version mismatch when I try and import it with anything but that linked system python. Why does the distutils module work fine without being linked? And if thats the case, why do I need to have the cmake build linking libpython, and how could I prevent it if thats the case so that it works with any of my python2.7 interpreters without a crash? Currently I can direct the cmake at the right python with: `CXX=g++ cmake -DPYTHON_LIBRARY=/path/to/another/Python`

Original source