[Xapian-discuss] flintlock fork causes hang in Apache+Python+mod_python

[Richard Boulton]

Deron Meranda wrote:
> I am trying to use Xapian 1.0.2 with the Python SWIG
> bindings withn an environment consisting of Apache
> httpd with mod_python.  (not as a CGI)  Also this is
> Linux.
> 
> Whenever the python code attempts to open a database
> the entire httpd process will hang indefinitely.  The
> python bindings work outside of the apache/mod_python
> environment.

I think this is probably a slightly different problem which was 
identified on the list (or maybe it was just on the bug tracker) a 
little while ago.  I added a piece of documentation on how to work 
around it to the SVN tree a little while ago, but it's not in a released 
version yet.


So you don't have to check out SVN, the documentation I wrote is as follows:

There is an issue with python's threading support and mod-python,
which can cause problems when using the Xapian Python bindings
from mod-python.  If you find that your mod-python application
is hanging whenever a Xapian method is called, you have probably
run into this problem.

By default, mod-python uses a separate instance of the python 
interpreter for
each mod-python application.  However, the Xapian python bindings do not 
know
how to get and release control of the thread lock unless they are running in
the "main" python interpreter; as a result, deadlocks can occur.

To avoid the problem, you need to set the "PythonInterpreter 
main_interpreter"
option in the apache configuration section for all mod_python scripts 
which use
xapian: this ensures that the xapian bindings are called from the
main_interpreter, and can handle the thread locks appropriately.  An example
configuration file using this option is available <a
href="http://www.xapian.org/cgi-bin/bugzilla/attachment.cgi?id=121&action=view">here</a>.

In the longer term, we hope to be able to resolve this problem, so that
arbitrary python interpreters can be used; but this is quite a tricky 
problem
to fix, and is relatively low priority.  More details can be found in 
Xapian's
bug tracker, as bug #185:
http://www.xapian.org/cgi-bin/bugzilla/show_bug.cgi?id=185

-- 
Richard