Click-through measurements are certainly good measure for automatic preparation of training data. But what I have in my mind is if we consider relevance as a binary variable then For the training data there are many relevance judgements are available for ad-hoc retrieval task in many good IR conferences like TREC or FIRE, so we can prepare the feature vectors from them. It will be a first benchmark for the project guideline. It will be reliable too because it is human-judged and comprises both the relevant and non-relevant documents. So an unbiased sample and good for machine learning.<br>
<br>Sure I am certainly very happy to discuss it with you, because thats how I can convey my idea well, through answering questions.<br><br>Also I am very new to the formalities to submit the application for the GSoC so if the things happen early then I would have enough time to shape the application considering feedbacks.<br>
<br>Cheers,<br>Parth.<br><br><div class="gmail_quote">On Sun, Apr 3, 2011 at 8:10 PM, Olly Betts <span dir="ltr"><<a href="mailto:olly@survex.com">olly@survex.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im">On Fri, Apr 01, 2011 at 02:48:28PM +0530, Parth Gupta wrote:<br>
> In Laarning to Rank (Letor) we prepare the features which can represent a<br>
> query document pair. So now after the initial retrieval we take say first 20<br>
> or 30 documents and represent them in form of feature vactors, now based on<br>
> the training data our supervised leaning will give a score to each document<br>
> for a particular query. For example if this learning is from regression then<br>
> we have to learn 'W' vector which will give a score to the document vector<br>
> by dot product.<br>
><br>
> Here the features can be term frequency, TF-IDF score, BM25 Score etc, as<br>
> good as many. For Learning there are many machine learning techniques<br>
> available.<br>
<br>
</div>What would be your plan for gathering data to train with? Some sort of<br>
click-through measurements?<br>
<div class="im"><br>
On Sun, Apr 03, 2011 at 12:37:27PM +0530, Parth Gupta wrote:<br>
> Please give your feedback on the possibility of exploration of the idea so<br>
> that I can incorporate those things in my application.<br>
<br>
</div>It seems an interesting project to me, though I'm not sure I know enough<br>
about the are to offer a much in the way of useful insights. I can<br>
probably ask some stupid questions though.<br>
<br>
But I'm certainly happy to consider an application from you for working<br>
on this.<br>
<br>
Cheers,<br>
<font color="#888888"> Olly<br>
</font></blockquote></div><br>