[Xapian-discuss] hypens in words + NEAR + 3 terms + AND_MAYBE => crash

[Marinos Yannikos]

Probably an uncaught malformed query - the following form of search queries 
causes a crash for me (core 1.2.3, Perl API, 64bit Debian Lenny, self-compiled):

  x-y NEAR test NEAR test

The first term can be anything with a hyphen in it but word characters at the 
beginning and end ("3--3" will do). The other 2 terms can be anything.

"test NEAR x-y NEAR test" will not cause a crash, but "test NEAR test NEAR x-y" 
will.

Stemming language was "english", strategy STEM_SOME. Default op was OP_AND_MAYBE.

This is the offending Perl code:

[...]
     my $qp = Search::Xapian::QueryParser->new();
     my $stemmer = Search::Xapian::Stem->new("english");
     $qp->set_stemmer($stemmer);
     $qp->set_database($database);
     $qp->set_stemming_strategy(STEM_SOME);
     $qp->set_default_op(OP_AND_MAYBE);

     $query_string = " x-y NEAR test NEAR test ";
     my $query = $qp->parse_query($query_string, Search::Xapian::FLAG_DEFAULT);
[...]

Here's a gdb backtrace for a crash:

Program received signal SIGSEGV, Segmentation fault.
~TermGroup (yypParser=0x2b58170, yymajor=<value optimized out>,
     yypminor=0x386a7f8) at queryparser/queryparser.lemony:1265
1265                delete *i;
Current language:  auto; currently c++
(gdb) bt
#0  ~TermGroup (yypParser=0x2b58170, yymajor=<value optimized out>,
     yypminor=0x386a7f8) at queryparser/queryparser.lemony:1265
#1  TermGroup::destroy (yypParser=0x2b58170, yymajor=<value optimized out>,
     yypminor=0x386a7f8) at queryparser/queryparser.lemony:1256
#2  yy_destructor (yypParser=0x2b58170, yymajor=<value optimized out>,
     yypminor=0x386a7f8) at queryparser/queryparser.lemony:1933
#3  0x00007faab7501f0c in yy_pop_parser_stack (pParser=0x2b58170)
     at queryparser/queryparser_internal.cc:2039
#4  0x00007faab750cf58 in ParserHandler::~ParserHandler() ()
    from /usr/local/lib/libxapian.so.22
#5  0x00007faab7508e0d in Xapian::QueryParser::Internal::parse_query (
     this=0x38b7a90, qs=..., flags=0, default_prefix=<value optimized out>)
     at queryparser/queryparser.lemony:1176
#6  0x00007faab750003a in Xapian::QueryParser::parse_query (this=0x37b1a10,
     query_string=..., flags=135, default_prefix=...)
     at queryparser/queryparser.cc:118
#7  0x00007faab77cffcc in XS_Search__Xapian__QueryParser_parse_query (
     my_perl=<value optimized out>, cv=<value optimized out>)
     at ./XS/QueryParser.xs:58

It does not seem to depend on the contents of the index, since it already 
happens early while rebuilding it and it happens with 
xapian-core-1.2.3/tests/testdata/flint-1.0.2/ as the index too. With xapian-core 
1.0.18, the same code yields an error: "Exception: Exception: Xapian::Query: 
AND_MAYBE requires a minimum of 2 and a maximum of 2 sub queries, had 4"

Please let me know if there's anything else I can test/provide.

Regards,
  Marinos