[Xapian-devel] Learning to rank

Parth Gupta parthg.88 at gmail.com
Fri Mar 30 13:04:09 BST 2012


Hi Pankaj,

Nice to see that you have chosen the algorithm. Yes, indeed ListMLE would
be a nice choice hence the difference between ListNet and ListMLE is the
loss function. The former mimimises the Cross Entropy while the latter
miminises the likelihood loss.

It would be better, if you investigate this and try to include in your
proposal.

Parth.

> Here is the idea which i want to incorporate and which would be a good
> extension to the LTR project and Xapian.
> I want to implement the algorithm ListMLE[1] on Xapian. The algorithm uses
> listwise approach with Neural Network as Model and gradient descent as
> algorithm(highly optimised Loss function). ListMLE is an extension of
> ListNET[2] which itself is an extension(somewhat) of RankNET[2]. This
> algorithm has shown better performance than the other two.Also the
> algorithm has linear complexity.
>
> Regarding the features for the query-document pair, research has shown
> many good features that can be used for better tuning of the parameters of
> ranking function which can differentiate the documents in a better way.
> These can be calculated using the basic set of features(tf, idf, bm25,
> etc.), the more the better.
>
> Regarding the training data we can use the OHSUMED[4] data-set, a
> benchmark data-set released in LETOR 2.0(Microsoft research), used by the
> developers of the algorithm for the training and testing purposes. This
> data-set is reliable as the relevance degrees of documents with respect to
> the queries are judged by humans. They try to adopt the ‘standard’ features
> proposed in the IR community. The similar kind of features, as used in
> data-set, can be incorporated while implementing the algorithm on Xapian.
>
> Implementing this algorithm would definitely be a good improvement in the
> current LTR project, as it uses listwise approach which is far better than
> the current pointwise approach. Also there are more and better features
> used in OHSUMED dataset which we can use , than the current used features.
>
> Please give feedback on the idea and suggest any exploration needed.
>
>
> [1] - http://research.microsoft.com/en-us/people/tyliu/icml-listmle.pdf
> [2] - http://research.microsoft.com/apps/pubs/default.aspx?id=70428
> [3] -
> http://research.microsoft.com/en-us/um/people/cburges/papers/ICML_ranking.pdf
> [4] -
> http://research.microsoft.com/en-us/um/beijing/projects/letor//letor-old.aspx
>
>
> regards,
>
>
> On Wed, Mar 28, 2012 at 7:58 PM, Parth Gupta <parthg.88 at gmail.com> wrote:
>
>> Pankaj,
>>
>> FANN looks fine. But in the proposal I would like to see something
>> specific what you plan to do with that. Like implementing the algorithm
>> RankNet, ListNet or something else?
>>
>> Parth.
>>
>>
>> On Wed, Mar 28, 2012 at 6:19 AM, Olly Betts <olly at survex.com> wrote:
>>
>>> On Tue, Mar 27, 2012 at 05:26:45PM +0530, pankaj singhal wrote:
>>> > I have come across these C++ neural-frameworks:
>>> > FANN <http://leenissen.dk/fann/wp/>
>>> > Libann <http://www.nongnu.org/libann/doc/libann_4.html#SEC17>
>>>
>>> Did you check the licences?  Libann's site clearly says it's GPL and as
>>> I said in the message you replied to, we'd rather not add more GPL
>>> dependencies.
>>>
>>> > I want you to look at the libraries as while incorporating them the
>>> need of
>>> > implementing the ML algo. from the scratch reduces.
>>> > http://lists.xapian.org/mailman/listinfo/xapian-devel
>>>
>>> FANN says it is LGPL, which is probably OK.  I've no idea if it fulfils
>>> the needs of the project.  Parth may be able to comment more usefully,
>>> but ultimately you'll need to show us in your proposal that the
>>> libraries you're intending to use are suitable, so you'll need to look
>>> into this more deeply yourself.
>>>
>>> Cheers,
>>>     Olly
>>>
>>
>>
>
>
> --
> Pankaj Singhal
> III Year, CSE
> The LNMIIT, Jaipur, India.
>
> Mob: +918875053936
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20120330/82e9af5a/attachment-0001.htm>


More information about the Xapian-devel mailing list