[Xapian-tickets] [Xapian] #185: Deadlocks with apache mod_python and mod_wsgi

[Xapian]

#185: Deadlocks with apache mod_python and mod_wsgi
-----------------------------+----------------------------------------------
 Reporter:  richard          |        Owner:  richard 
     Type:  defect           |       Status:  assigned
 Priority:  normal           |    Milestone:          
Component:  Xapian-bindings  |      Version:  SVN HEAD
 Severity:  normal           |   Resolution:          
 Keywords:                   |    Blockedby:          
 Platform:  All              |     Blocking:          
-----------------------------+----------------------------------------------

Comment(by grahamd):

 I'll have to do some more research as my memory is hazy. That issue though
 really has two parts to it. The first is the bit that only affects Python
 2.3 and the second is the general problem of using using simplified GIL
 state API. If is the latter part which affects SWIG bindings such as
 Xapian. The first part refers only to a particular scenario which I don't
 think can occur for SWIG bindings but is more an issue for mod_python
 internals.

 When using SWIG bindings there are normally two use case scenarios. The
 first is:

   1. Python code executing and GIL held.
   2. SWIG binding leaves Python and releases GIL but remembers its
 original thread state.
   3. C code is executed.
   4. C code finishes and SWIG binding restores original thread state and
 returns back to calling Python code.

 The second is:

   1. Python code executing and GIL held.
   2. SWIG binding leaves Python and releases GIL but remembers its
 original thread state.
   3. C code is executed.
   4. C code wants to make callback into Python and uses simplified GIL
 state API to reacquire GIL.
   5. Python code executes and returns.
   6. GIL released using simplified GIL state API.
   7. C code finishes and SWIG binding restores original thread state and
 returns back to calling Python code.

 It is the second of these which fails in mod_python even when main
 interpreter due to mod_python not using simplified GIL state API for all
 main interpreter code.

 The first case is safe in all interpreters as not involving simplified GIL
 state API.

 The Python 2.3 specific scenario is:

   1. C code executing and active thread state still exists with GIL
 locked.
   2. Simplified GIL state API called to try and acquire lock again.

 This final case I don't believe can arise with SWIG bindings being used
 from Python code, thus why I think Python 2.3 exclusion is not relevant to
 Xapian.

 BTW, not compiling SWIG bindings with thread support would preclude
 callbacks. Plus, if C code blocked for long time, would cause all of
 Python to block as GIL not released for case 1 above.

 More later maybe, as have to go some where now.

-- 
Ticket URL: <http://trac.xapian.org/ticket/185#comment:29>
Xapian <http://xapian.org/>
Xapian