GSoC 2017: Letor Click Data Mining

[Vivek Pal]

Hi Olly. Thanks for your reply to the previous email.

To have an appropriate subject I've started this new thread for further
discussions.

> There's a $log{} command available in Omega templates. We can't log from
> the result page template, as the clicks happen after that is used, but we
> could make result links redirect via a second Omega template which does
> the logging.

To make sure I understand correctly -- we need a second Omega template to
enable
redirection of each result link from opensearch template to that template
after they are
clicked, our first step should be towards implementing such a template or
it exists already?
Please correct me if I'm wrong.

I've been exploring the Omega codebase for the past few days but it would
be great if
you could elaborate a bit about how logging works internally. So far, I
understand that
$log command recursively calls the eval function in which it's defined and
prints the
returned string from eval to a log file.

> So for that you'd also need to implement this result modification and then
> to use that new feature from Omega.

I've read the paper to understand the FairPairs algorithm. To implement it,
we'd need to
take the result links for each query from opensearch template and feed it
into the algorithm
which will rearrange the results using a uniform probability variable.
Modified results can
then presented using opensearch template and clicks are recorded to adjust
the relevance
score of certain doc links.

> That indeed seems an unrealistic assumption, though I guess what really
> matters is how effective these models are in practice (after all, models
> are almost inherently simplifications of reality).

I'm planning to propose preference pairs model along with FairPairs to
eliminate position
bias. Do you suggest to look into one of the sequential models as well?

Also, I'm following the Omega example wiki page to setup Omega at present
to have a first
hand experience with it. I've xapian-core and Omega installed on my system
(had it installed
earlier but pulled the recent changes and installed again).

But visiting localhost/cgi-bin/omega.cgi gives 500 Internal Sever Error.
Looking into error log
file (https://paste.debian.net/922929/) reveals that it doesn't have
permission to create .libs
directory in /usr/lib/cgi-bin which shouldn't be the case as I set the
permissions correctly using:

sudo chmod 755 /usr/lib/cgi-bin/omega.cgi

Also, I didn't see the below output while indexing the data using omnidex.

[Entering directory /]
Indexing "/ci_01.htm" as text/html ...  added.
Indexing "/ci_02.htm" as text/html ...  added.
...

Did I miss something in the process?

Thanks,
Vivek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20170321/0eca9da2/attachment.html>