[Xapian-devel] Complete GSOC idea

[Olly Betts]

On Tue, Mar 04, 2014 at 08:58:57PM +0530, Aarsh Shah wrote:
> Also, is the evaluation module fully functional ? I saw that some
> issues are still open on it.

I think most of the issues are just stuff to do with the build system,
or fairly minor things, but I haven't looked at it recently.  The plan
is to try to get it merged soon, but the LMWeight stuff is up first.

> Also, I initially thought I would write the query log and expected
> results set by hand for some wikipedia articles but realize now that
> you have a point as we need to test on a large number of articles.

And the "expected results" you ideally need are a decision on whether
each document in the collection is relevant to each query or not.

You probably only really need judgements for the documents returned
by a particular query in any of the tests you do (i.e. if, for a given
query, if there's a relevant document which never gets returned by any
weighting scheme under test, that can probably just be ignored).  But
that's still a lot of judgements.

Cheers,
    Olly