[Xapian-tickets] [Xapian] #696: Segfault/OOM in git master

[Xapian]

#696: Segfault/OOM in git master
-----------------------------------------------+---------------------------
 Reporter:  coventry                           |             Owner:
                                               |  richard
     Type:  defect                             |            Status:  new
 Priority:  normal                             |         Milestone:  1.3.x
Component:  Xapian-bindings (Python)           |           Version:  other
 Severity:  major                              |        Resolution:
 Keywords:  1.3.3 segfault git master 9c9213d  |        Blocked By:
 Blocking:                                     |  Operating System:  Linux
-----------------------------------------------+---------------------------

Old description:

> I don't have time to reduce this to a minimal test case, but the process
> for reproducing the failure is automated, so even though it takes a long
> time to run, it's not too much human effort.
>
> Start up a $10 Ubuntu 14.04 digital ocean box, and run the attached bash
> script.  Takes a couple of hours to complete.  The final python command
> either segfaults or gets killed for excessive memory usage.  Usually
> segfaults, in my experience.  With a bit of print debugging, I've
> determined it's segfaulting on "enquire.get_mset(0, numresults)",
> unsurprisingly.
>
> Here are the versions of all the git checkouts during the build process.
>
> root at testscript:~/voter_lookup# for p in `find /root -name .git -type d`
> ; do ( cd $p ; git log -1 --pretty=format:"%h $p" ) ; done[[BR]]
> 94ca849 /root/voter_lookup/.git[[BR]]
> 1a99212 /root/alien/xapian/swig/.git[[BR]]
> 3c0adf0 /root/alien/xapian/xapian-
> applications/omega/.common.git/.git[[BR]]
> 3c0adf0 /root/alien/xapian/xapian-letor/.common.git/.git[[BR]]
> 9c9213d /root/alien/xapian/.git

New description:

 I don't have time to reduce this to a minimal test case, but the process
 for reproducing the failure is automated, so even though it takes a long
 time to run, it's not too much human effort.

 Start up a $10 Ubuntu 14.04 digital ocean box, and run the attached bash
 script.  Takes a couple of hours to complete.  The final python command
 either segfaults or gets killed for excessive memory usage.  Usually
 segfaults, in my experience.  With a bit of print debugging, I've
 determined it's segfaulting on "enquire.get_mset(0, numresults)",
 unsurprisingly.

 Here are the versions of all the git checkouts during the build process.

 {{{
 root at testscript:~/voter_lookup# for p in `find /root -name .git -type d` ;
 do ( cd $p ; git log -1 --pretty=format:"%h $p" ) ; done
 94ca849 /root/voter_lookup/.git
 1a99212 /root/alien/xapian/swig/.git
 3c0adf0 /root/alien/xapian/xapian-applications/omega/.common.git/.git
 3c0adf0 /root/alien/xapian/xapian-letor/.common.git/.git
 9c9213d /root/alien/xapian/.git
 }}}

--

Comment (by olly):

 My guess is it's simply exceeding the allocated stack depth, which
 typically leads to a segfault - I don't have an Ubuntu machine within
 reach, but on Debian x86_64:

 {{{
 $ cat > stack.cc
 int f(int c) {
    return f(c + 1) + f(c + 2);
 }

 int main() { return f(0); }
 $ g++ stack.cc
 $ ./a.out
 Segmentation fault
 }}}

 You say it sometimes fails with OOM, which fits that conclusion.

 What does `free` report for this VM?  How big is the database?

 I'm afraid a reproducer which involves having to buy and set up a special
 VM and then wait several hours per iteration doesn't exactly fill me with
 the urge to dig in...

--
Ticket URL: <http://trac.xapian.org/ticket/696#comment:3>
Xapian <http://xapian.org/>
Xapian