[Xapian-tickets] [Xapian] #623: _xapian.so shoud link against libpythonX.X.so

[Xapian]

#623: _xapian.so shoud link against libpythonX.X.so
--------------------------------------+----------------------------
 Reporter:  lilydjwg                  |             Owner:  richard
     Type:  defect                    |            Status:  new
 Priority:  normal                    |         Milestone:
Component:  Xapian-bindings (Python)  |           Version:  1.3.1
 Severity:  normal                    |        Resolution:
 Keywords:                            |        Blocked By:
 Blocking:                            |  Operating System:  All
--------------------------------------+----------------------------

Comment (by olly):

 > The need to support python 2 and 3 is interesting; I think that would
 make it harder (or impossible) for vim to link the necessary symbols in
 itself, so its more reasonable that we should be linking to libpython
 ourself.

 FWIW, that's probably not an inherent limitation - if vim passed
 RTLD_GLOBAL to dlopen() when it loaded whichever version of libpython the
 user wants to use, then the symbols from libpython would be available to
 anything that gets loaded by dlopen() afterwards.  It probably couldn't
 support actually loading libpython2 and libpython3 at once (at least if
 they use some of the same symbols).

 Anyway, we don't use python-config at all currently.  I suspect it didn't
 even exist when the configure checks in xapian-bindings were written -
 looking at older debian packages, I see it in python-2.5 but not
 python-2.4.  http://bugs.python.org/issue1161914 seems to be the ticket
 for adding it, but doesn't mention versions that I can see.  It doesn't
 seem to be mentioned by any python docs apart from its own --help output
 and man page, so its existence isn't all that easy to discover.

 So, python-config's docs suggest:

 {{{
        To build a dynamically loadable python module, use

               gcc $(python-config --cflags --ldflags) -shared -fPIC
 progr.cpp -o progr.so
 }}}

 Which sounds good, but its --ldflags option conflates "linker arguments to
 use when embedding python into a program" with "linker arguments to use
 when linking a python module".  I think distutils is a more reliable
 source for how such a module should be linked, and for the simple example
 from http://docs.python.org/2/extending/building.html on Debian wheezy I
 get (with demo.c just being an empty file):

 {{{
 $ python setup.py build_ext
 running build_ext
 building 'demo' extension
 creating build
 creating build/temp.linux-x86_64-2.7
 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-
 prototypes -fPIC -I/usr/include/python2.7 -c demo.c -o build/temp.linux-
 x86_64-2.7/demo.o
 creating build/lib.linux-x86_64-2.7
 gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro
 build/temp.linux-x86_64-2.7/demo.o -o build/lib.linux-x86_64-2.7/demo.so
 }}}

 Note that it doesn't link with libpython.

 Interestingly, I checked the installed python modules on that machine and
 '''none''' of them are linked to libpython:

 {{{
 $ find /usr/lib/python2.7 -name '*.so' -print | wc -l
 86
 $ find /usr/lib/python2.7 -name '*.so' -print0 | xargs -0 -n1 ldd |grep
 -cw libz
 34
 $ find /usr/lib/python2.7 -name '*.so' -print0 | xargs -0 -n1 ldd |grep
 libpython
 $
 }}}

 As richard says, on Windows we do link the compiled module with libpython
 - that's because a DLL can't have undefined symbols which get resolved by
 the application loading it (or some limitation like that).  I don't have a
 windows machine to test on, but I'd expect distutils to link to libpython
 there.

 > I wasn't actually advocating closing, I was trying to get more
 information to work out if this is something we should be changing.

 Yeah, it would definitely be good to sort out what's expected from
 extension modules here.

--
Ticket URL: <http://trac.xapian.org/ticket/623#comment:6>
Xapian <http://xapian.org/>
Xapian