[Xapian-discuss] Xapian 1.0 / Apache / Python problem

[James Aylett]

On Fri, May 25, 2007 at 02:19:19PM +0100, Richard Boulton wrote:

> >By default mod_python prints stack traces. There's no exception being 
> >fired, at least not in Python-space.
> 
> So, it's just hanging for an indefinite time.   Sounds like a threading 
> problem, perhaps.

Is it hanging, or is it crashing? Does the apache child survive? If
it's a segfault, you'll probably get a notification in the apache core
(ie non-virtual server) error log, but nothing to tie that to the
request.

> >What's still bugging me is that this only happens when running under 
> >apache. Is there a way to run a debugger in that circumstance?
> 
> I'm afraid I don't know about the debugger part, but I'm suspecting that 
> this is something to with a change we made to make the Xapian Python 
> bindings drop the GIL when calling the C library, to enable concurrent 
> searches and other operations.

You can attach a debugger to the running apache process. Configure
apache to run one child, pre-fork, no maxrequestsperchild limit, fire
a couple of neutral requests at it, figure out the process id and
attach gdb or debugger-of-choice to that. (It'll be the non-root
process, probably.)

If you don't have debugging symbols compiled in, this might not help
much. strace / truss may help you here (it'll output the syscalls
being made, in case there's something obvious there). Alternatively,
you could look at DTrace (Solaris) or systrap (linux).

> I'm guessing that the threading environment provided when you're called 
> from apache is incompatible in some way with the threading stuff we're 
> doing.  We're using Python's threading functions though, so I'm quite 
> surprised by this.

I'm not *overly* surprised, given that Python doesn't really
understand threads...

J

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james at tartarus.org                               uncertaintydivision.org