[Xapian-devel] xapian csharp bindings crash

[Greg]

On Tue, Jun 28, 2011 at 8:09 PM, Greg <freediving at gmail.com> wrote:
> On Tue, Jun 28, 2011 at 5:49 PM, Olly Betts <olly at survex.com> wrote:
>>> ...
>>> I've actually wondered about it myself but I thought that it was
>>> called internally for some reason.
>>No, nothing calls TermGenerator implicitly.
>
> Which is extremely odd, the only explanation I can find would be that
> the debugger (windbg) didn't interpret the exception stack trace
> correctly.
>
>>> using (var xdb = OpenQueryDatabase(DBtype.AutoCompleteThreads))
>>
>> Nothing looks obviously wrong here, but the mention of threads makes
>> me wonder - are you perhaps sharing Xapian objects between threads?
>> If so, it's your responsibility to ensure they aren't used from two
>> different threads at once.  Either you need a mutex or similar, or
>> just to use a different object in each thread.  Failing to do this
>> can cause weird behaviour, which potentially fits with what you are
>> seeing.
>
> Nothing as sinister as that, the Threads reference in the name
> actually relates to messages and threads of messages i.e. it's a forum
> application. We're actually creating Xapian objects for every request
> so they aren't shared (I'll double check with the OpenQueryDatabase
> method), which is probably not the best of ideas in terms of
> connections e.g. making some sort of a connection pooling class would
> probably help but that's fluff at the moment.
>
> And thank you very much for taking the time to answer my questions,
> I'll keep this updated if I have any more news on the issue.
>
I've been trying to get to the bottom of this, had to enable page heap
to try and debug this. What I did manage to get to was the following
line in xapian_wrap.cc @4996
result = (bool)Xapian_MSetIterator_equals((Xapian::MSetIterator const
*)arg1,(Xapian::MSetIterator const &)*arg2);

At least this is where the debugger complained about access violation,
on the surface it doesn't look like there is something going on, but I
was thinking this line is called from our code
 while (m != matches.End())
       {
              //do something with m
               ++m;
       }

What would happen if for some reason we increment [m] beyond the
bounds, I tried testing it by removing the check and it seemed to
continue incrementing [m] 'ad-infinitum?' without a crash I assume
it's possibe for it to cause a crash. Now this is using the assumption
that for some reason matches.End() is wrong because in the same
debugging session I checked the pointers arg1 and arg2 and at the
instance of the crash they were identical i.e. the same pointer.

The trouble is that the test runs take a very long time to reproduce
the crash, it doesn't crash in the same place in the work queue, it
does crash as in the crash is reproducible just not in a consistent
location which probably means it has nothing to do with the searches
executed but rather due to some memory allocation / freeing magic
going on within SWIG?