[Xapian-tickets] [Xapian] #185: Deadlocks with apache mod_python and mod_wsgi

[Xapian]

#185: Deadlocks with apache mod_python and mod_wsgi
-----------------------------+----------------------------------------------
 Reporter:  richard          |        Owner:  richard 
     Type:  defect           |       Status:  assigned
 Priority:  normal           |    Milestone:          
Component:  Xapian-bindings  |      Version:  SVN HEAD
 Severity:  normal           |   Resolution:          
 Keywords:                   |    Blockedby:          
 Platform:  All              |     Blocking:          
-----------------------------+----------------------------------------------

Comment(by richard):

 The Python 2.3 problem definitely did exist with Xapian, and I think it
 still does: last year, when this issue was first opened, I observed it
 failing as described, and the Xapian code is exactly as described in the
 http://issues.apache.org/jira/browse/MODPYTHON-217.  To pick a random
 example, the wrapper function for xapian.version_string() begins:

 {{{
 SWIGINTERN PyObject *_wrap_version_string(PyObject *SWIGUNUSEDPARM(self),
 PyObject *args) {
   PyObject *resultobj = 0;
   char *result = 0 ;

   SWIG_PYTHON_THREAD_BEGIN_BLOCK;
 }}}
 and SWIG_PYTHON_THREAD_BEGIN_BLOCK eventually expands to a call to
 PyGILState_Ensure().  This is called with the interpreter holding the GIL,
 and the situation is the same for pretty much every SWIG generated method,
 so any call to xapian results in deadlock.

 I don't have a Python 2.3 installation to hand to test this now, but the
 Xapian code hasn't changed, and I assume the mod-python code hasn't
 either, since the MODPYTHON-217 issue in the apache tracker is still open.
 I therefore believe the summary in this bug about the situation with 2.3
 is correct.

 With python 2.4, it is still necessary to set the interpreter to
 main_interpreter, as described at length in
 http://issues.apache.org/jira/browse/MODPYTHON-77 - this is because any
 use of PyGILState_Ensure() assumes that the interpreter running is the
 main interpreter.  Also, as you describe in your comment, callbacks to
 Python from C still won't work.  I think our "Summary of current known
 status" is therefore fully up-to-date and correct.

 I don't understand your comment "BTW, not compiling SWIG bindings with
 thread support would preclude callbacks."  If swig is told to generate the
 bindings without thread support, the resulting code doesn't touch the GIL
 at any point, so the callbacks should work fine (the lock is never
 released, or re-acquired, but the thread state isn't accessed either).  I
 fully agree with "Plus, if C code blocked for long time, would cause all
 of Python to block as GIL not released for case 1 above", though!

-- 
Ticket URL: <http://trac.xapian.org/ticket/185#comment:31>
Xapian <http://xapian.org/>
Xapian