[Xapian-devel] FLAG_PARTIAL and subset words

[Greg]

On Mon, May 16, 2011 at 9:34 AM, Greg <freediving at gmail.com> wrote:
> On Sun, May 15, 2011 at 10:57 PM, Olly Betts <olly at survex.com> wrote:
>> On Sun, May 15, 2011 at 11:25:04AM +0200, Greg wrote:
>>> We're using it this way:
>>> xq = qp.ParseQuery(query, (uint)Xapian.QueryParser.feature_flag.FLAG_PARTIAL);
>>> The op is AND and the get_description is fairly long (I removed some
>>> of it since there is a limit of 40k in the list), v90 is part of the
>>> SYNONYMs being generated:
>>
>> Then it isn't ignoring the second partial word at all, but expanding it
>> as it should.
>>
>> I think you've just found cases where it happens that the partial word
>> doesn't affect the results (or at least the top N that you're seeing).
>> That the partial word is a prefix to the other word in the query in both
>> cases just seems to be a coincidence!
>>
>> Cheers,
>>    Olly
> Not really, I've checked various queries in the same theme and the
> results are repeatable. Searching for "nikon n" just returns
> everyhting related to nikon "nikon f" on the other hand works as
> expected and vice versa "ford f" and "ford n".
>
> I'll try and create a new database containing only 2 items "volvo v90"
> and "volvo s60" as far as I understand searching for "volvo v" should
> return one result and "volvo" two.

Ok, a way to reproduce it; first the database is here:
http://pastebin.com/raw.php?i=sqCuRJnW (base64 of a 7zip containing
the db directory).

I'll paste the C# code we're using but it should be easily understandable.
Try with the xquery being "volvo", "volvo v", "volvo s"

Xapian.Database xdb = new
Database("c:\\Projects\\Xapian\\Indexer\\bin\\Debug\\products");
Xapian.Enquire enquire = new Enquire(xdb);
Xapian.QueryParser qp = new QueryParser();
Xapian.Stem stemmer = new Stem("english");

qp.SetStemmer(stemmer);
qp.SetDatabase(xdb);
qp.SetDefaultOp(Query.op.OP_AND);
qp.SetStemmingStrategy(Xapian.QueryParser.stem_strategy.STEM_SOME);
Xapian.Query xquery = qp.ParseQuery(query,
(uint)Xapian.QueryParser.feature_flag.FLAG_PARTIAL);
enquire.SetQuery(xquery);
Xapian.MSet matches = enquire.GetMSet(0, 3);
Xapian.MSetIterator m = matches.Begin();

while(m != matches.End())
{
	Console.WriteLine("{0}: {1}% docid={2} [{3}]\n",
	m.GetRank() + 1,
	m.GetPercent(),
	m.GetDocId(),
	m.GetDocument().GetData());

	++m;
}