Hi Pankaj,<br><br>Nice to see that you have chosen the algorithm. Yes, indeed ListMLE would be a nice choice hence the difference between ListNet and ListMLE is the loss function. The former mimimises the Cross Entropy while the latter miminises the likelihood loss. <br>
<br>It would be better, if you investigate this and try to include in your proposal.<br><br>Parth.<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Here is the idea which i want to incorporate and which would be a good extension to the LTR project and Xapian.<br>I want to implement the algorithm ListMLE[1] on Xapian. The algorithm uses listwise approach with Neural Network as Model and gradient descent as algorithm(highly optimised Loss function). ListMLE is an extension of ListNET[2] which itself is an extension(somewhat) of RankNET[2]. This algorithm has shown better performance than the other two.Also the algorithm has linear complexity.<br>
<br>Regarding the features for the query-document pair, research has shown many good features that can be used for better tuning of the parameters of ranking function which can differentiate the documents in a better way. These can be calculated using the basic set of features(tf, idf, bm25, etc.), the more the better.<br>
<br>Regarding the training data we can use the OHSUMED[4] data-set, a benchmark data-set released in LETOR 2.0(Microsoft research), used by the developers of the algorithm for the training and testing purposes. This data-set is reliable as the relevance degrees of documents with respect to the queries are judged by humans. They try to adopt the ‘standard’ features proposed in the IR community. The similar kind of features, as used in data-set, can be incorporated while implementing the algorithm on Xapian.<br>
<br>Implementing this algorithm would definitely be a good improvement in the current LTR project, as it uses listwise approach which is far better than the current pointwise approach. Also there are more and better features used in OHSUMED dataset which we can use , than the current used features. <br>
<br>Please give feedback on the idea and suggest any exploration needed.<br><br><br>[1] - <a href="http://research.microsoft.com/en-us/people/tyliu/icml-listmle.pdf" target="_blank">http://research.microsoft.com/en-us/people/tyliu/icml-listmle.pdf</a><br>
[2] - <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=70428" target="_blank">http://research.microsoft.com/apps/pubs/default.aspx?id=70428</a><br>[3] - <a href="http://research.microsoft.com/en-us/um/people/cburges/papers/ICML_ranking.pdf" target="_blank">http://research.microsoft.com/en-us/um/people/cburges/papers/ICML_ranking.pdf</a><br>
[4] - <a href="http://research.microsoft.com/en-us/um/beijing/projects/letor//letor-old.aspx" target="_blank">http://research.microsoft.com/en-us/um/beijing/projects/letor//letor-old.aspx</a><br><br><br>regards,<div><div>
</div><div class="h5"><br><br><div class="gmail_quote">
On Wed, Mar 28, 2012 at 7:58 PM, Parth Gupta <span dir="ltr"><<a href="mailto:parthg.88@gmail.com" target="_blank">parthg.88@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Pankaj,<br><br>FANN looks fine. But in the proposal I would like to see something specific what you plan to do with that. Like implementing the algorithm RankNet, ListNet or something else?<span><font color="#888888"><br>
<br>Parth.</font></span><div><div><br><br><div class="gmail_quote">
On Wed, Mar 28, 2012 at 6:19 AM, Olly Betts <span dir="ltr"><<a href="mailto:olly@survex.com" target="_blank">olly@survex.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On Tue, Mar 27, 2012 at 05:26:45PM +0530, pankaj singhal wrote:<br>
> I have come across these C++ neural-frameworks:<br>
</div>> FANN <<a href="http://leenissen.dk/fann/wp/" target="_blank">http://leenissen.dk/fann/wp/</a>><br>
> Libann <<a href="http://www.nongnu.org/libann/doc/libann_4.html#SEC17" target="_blank">http://www.nongnu.org/libann/doc/libann_4.html#SEC17</a>><br>
<br>
Did you check the licences? Libann's site clearly says it's GPL and as<br>
I said in the message you replied to, we'd rather not add more GPL<br>
dependencies.<br>
<div><br>
> I want you to look at the libraries as while incorporating them the need of<br>
> implementing the ML algo. from the scratch reduces.<br>
</div>> <a href="http://lists.xapian.org/mailman/listinfo/xapian-devel" target="_blank">http://lists.xapian.org/mailman/listinfo/xapian-devel</a><br>
<br>
FANN says it is LGPL, which is probably OK. I've no idea if it fulfils<br>
the needs of the project. Parth may be able to comment more usefully,<br>
but ultimately you'll need to show us in your proposal that the<br>
libraries you're intending to use are suitable, so you'll need to look<br>
into this more deeply yourself.<br>
<br>
Cheers,<br>
<font color="#888888"> Olly<br>
</font></blockquote></div><br>
</div></div></blockquote></div><br><br clear="all"><br></div></div><div><div></div><div class="h5">-- <br>Pankaj Singhal<br>III Year, CSE<br>The LNMIIT, Jaipur, India.<br><br>Mob: <a href="tel:%2B918875053936" value="+918875053936" target="_blank">+918875053936</a><br>
<br><br><br>
</div></div></blockquote></div><br>