[Xapian-devel] xapian csharp bindings crash

[Olly Betts]

On Thu, Jul 07, 2011 at 11:29:14AM +0200, Greg wrote:
> I've been trying to get to the bottom of this, had to enable page heap
> to try and debug this. What I did manage to get to was the following
> line in xapian_wrap.cc @4996
> result = (bool)Xapian_MSetIterator_equals((Xapian::MSetIterator const
> *)arg1,(Xapian::MSetIterator const &)*arg2);

If you can get this up in an interactive debugger, then it might be
interesting to see what *arg1 and *arg2 look like.

> At least this is where the debugger complained about access violation,
> on the surface it doesn't look like there is something going on, but I
> was thinking this line is called from our code
>  while (m != matches.End())
>        {
>               //do something with m
>                ++m;
>        }
> 
> What would happen if for some reason we increment [m] beyond the
> bounds, I tried testing it by removing the check and it seemed to
> continue incrementing [m] 'ad-infinitum?' without a crash I assume
> it's possibe for it to cause a crash.

The way MSetIterator is currently implemented is an MSet object (which
is just a reference counted pointer really) and an index into it.

So you can increment one forever and it actually just increments a
counter.  It's unsigned, so it'll just wrap eventually.  If you
try to *dereference* m once it is == matches.End() or beyond, then
you probably will get a crash.

And the end() iterator is the same thing, but the offset is mset.size().

It would be pretty easy to patch MSetIterator::operator * in
xapian-core's api/omenquire.cc to check that index is < mset.size()
if you wanted to see if this was happening.  I'll commit a change
to add assertions to these functions in a few minutes, so this
will be caught in a debug build in future (actually, if you aren't
already, running a build of Xapian with assertions enabled might help
track this down, though it is a fair bit slower with them all on).

> Now this is using the assumption
> that for some reason matches.End() is wrong because in the same
> debugging session I checked the pointers arg1 and arg2 and at the
> instance of the crash they were identical i.e. the same pointer.

That sounds odd.

> The trouble is that the test runs take a very long time to reproduce
> the crash, it doesn't crash in the same place in the work queue, it
> does crash as in the crash is reproducible just not in a consistent
> location which probably means it has nothing to do with the searches
> executed but rather due to some memory allocation / freeing magic
> going on within SWIG?

Perhaps, though SWIG's C# backend is one of the more actively maintained
ones.

If it was Linux, I'd say to point valgrind at it.  I don't know of
anything similar for MS Windows though.

Cheers,
    Olly