[Xapian-tickets] [Xapian] #185: Deadlocks with apache mod_python and mod_wsgi

[Xapian]

#185: Deadlocks with apache mod_python and mod_wsgi
-----------------------------+----------------------------------------------
 Reporter:  richard          |        Owner:  richard 
     Type:  defect           |       Status:  assigned
 Priority:  normal           |    Milestone:          
Component:  Xapian-bindings  |      Version:  SVN HEAD
 Severity:  normal           |   Resolution:          
 Keywords:                   |    Blockedby:          
 Platform:  All              |     Blocking:          
-----------------------------+----------------------------------------------

Old description:

> = Summary of current known status =
>
> == mod_python ==
>
> Calling any Xapian methods or functions is likely to cause dead-lock
> unless you set the option {{{PythonInterpreter main_interpreter}}} in the
> Apache configuration section for all mod_python scripts which use Xapian.
>
> You may also need to use Python >= 2.4 (due to
> [http://issues.apache.org/jira/browse/MODPYTHON-217 problems with the
> APIs used in Python 2.3).
>
> Even with this fixed, calling from Xapian back to Python code won't work
> properly (this means that you can't subclass Xapian objects in Python).
> This is apparently an issue with mod_python.
>
> == mod_wsgi ==
>
> You'll need to set:
>
> {{{
> WSGIApplicationGroup %{GLOBAL}
> }}}
>
> For details see:
> http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIApplicationGroup
> and
> http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API
>
> The mod_wsgi developers say this should be sufficient, and you should be
> able to subclass Xapian objects in Python.  If you encounter problems,
> please talk to us or the mod_wsgi developers so we can investigate.
>
> ---
>
> Originally reported on the mailing list:
> http://thread.gmane.org/gmane.comp.search.xapian.general/4486

New description:

 = Summary of current known status =

 == mod_python ==

 Calling any Xapian methods or functions is likely to cause dead-lock
 unless you set this option in the Apache configuration section for all
 mod_python scripts which use Xapian:

 {{{
 PythonInterpreter main_interpreter
 }}}

 You may also need to use Python >= 2.4 (due to
 [http://issues.apache.org/jira/browse/MODPYTHON-217 problems in Python 2.3
 with the APIs the code uses).

 Even with {{{main_interpreter}}} and Python >= 2.3, calling from Xapian's
 C++ code back to Python code won't work properly (this means that you
 can't subclass Xapian objects in Python).  This is apparently an issue
 with mod_python.

 == mod_wsgi ==

 You'll need to set:

 {{{
 WSGIApplicationGroup %{GLOBAL}
 }}}

 For details see:
 http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIApplicationGroup
 and
 http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API

 The mod_wsgi developers say this should be sufficient, and you should be
 able to subclass Xapian objects in Python.  If you encounter problems,
 please talk to us or the mod_wsgi developers so we can investigate.

 ----

 Originally reported on the mailing list:
 http://thread.gmane.org/gmane.comp.search.xapian.general/4486

--

Comment(by olly):

 Well, MODPYTHON-217 clearly says '''At this stage it is believed this only
 occurs with Python 2.3 and shouldn't be a problem with later versions of
 Python''' and there are no comments added which contradict this.  Here
 '''this''' refers to deadlock when calling {{{PyGILState_Ensure()}}} in
 the way in which SWIG generated wrappers do.  Is that wrong?

 And in comment:23 you say '''Thus, the suggestion of setting interpreter
 as 'main_interpreter' when using mod_python would definitely not work and
 lock ups would be seen if callbacks into Python from C code are occurring.
 If people are suggesting that it is working, they can't be using that
 feature of Xapian or have not compiled the SWIG bindings with thread
 support.'''  I understood that to mean that provided you don't use Xapian
 features which call from C++ back to Python, you can use Xapian bindings
 with thread support (which they have unless you regenerate them yourself
 without {{{-threads}}}).

 So either I've misunderstood here, or I've expressed this badly in my
 summary.  I've tweaked the wording in a couple of places - notably
 '''problems in Python 2.3 with the APIs the code uses''' - is that better?

-- 
Ticket URL: <http://trac.xapian.org/ticket/185#comment:28>
Xapian <http://xapian.org/>
Xapian