Hello,<div><br></div><div> I am a undergraduate student at DA-IICT,India pursuing Btech in Information and Communication Technology.Major field of my Research is Information Retrieval and Natural Language processing. xapain being an powerful Information retrieval library have attracted me towards implementing stuff learned in class for this project.I have worked on entity search on RDF data,SMS based FAQ retrieval,Question Answering under competitions in evaluation forums like CLEF ,FIRE.I want to grab GSOC opportunity and join world of FOSS developers.</div>
<div><br clear="all"><div>I would like to work and include hooping techniques like Language Modelling and Diversified Search in information retrieval.</div><div><br></div><div>Brief Summary of idea:</div><div><br></div><div>
Language Modelling for Information retrieval approach focus on building probabilistic language models for documents and rank document based on probability of model generating the query.Technique is heavy and costlier than the traditional information retrieval technique but has proved to preform better in literature than traditional methods.</div>
<div><br></div><div>Language modelling approach performs better as it tries to capture word and phrase association to capture user context.</div><div><br></div><div>Diversified search is key ways for user satisfaction in absence of explicit knowledge of user intent.Diversified search algorithm tries to find out(estimate) different possible context of user query and tries to pull potential document of all context rather than explicitly assuming a context.</div>
<div>Diversification can be done by generating different rank list for different context or adding document from different context in a single rank list.</div><div><br></div><div><div>Resources:</div><div><br></div><div>
<a href="http://nlp.stanford.edu/IR-book/html/htmledition/ponte-and-crofts-experiments-1.html" target="_blank">http://nlp.stanford.edu/IR-book/html/htmledition/ponte-and-crofts-experiments-1.html</a></div>
<div><a href="http://dl.acm.org/citation.cfm?id=291008" target="_blank">http://dl.acm.org/citation.cfm?id=291008</a></div></div><div><a href="http://goo.gl/klqYy" target="_blank">http://goo.gl/klqYy</a></div><div><a href="http://dl.acm.org/citation.cfm?id=1860709" target="_blank">http://dl.acm.org/citation.cfm?id=1860709</a></div>
<div><br></div>
<div><br></div><div>I have compiled and installed xapian and tried playing with xapian in past few days.I have few queries regarding xapian :- </div><div> </div><div>1. xapain supports relevance feedback(query expansion) through "<span style="background-color:rgb(213,225,232);font-family:'Lucida Grande',Verdana,Geneva,Arial,sans-serif;font-size:12px;font-weight:bold;white-space:nowrap">Xapian::Enquire::get_eset"</span> function.which algorithm is used to expand query in Enquire class. </div>
<div><br></div><div>Since search result diversification is its naive form performed by expanding query with different context and adding document from different context in final rank-list, thereby catering to all context of query.</div>
<div><br></div><div>I was thinking if i can use the algorithm implemented in expanded set for query expansion and implement a new algorithm in Search diversification in this way query expansion feature of xapian will also get powerful.</div>
<div>
<br></div><div>2. I have read that xapian supports passage retrieval ,proximity based query ,wildcard query and passage retrieval but I could not find any documentation or function providing these facilities of xapain.I will be glad if you can point me towards any available documentation describing to use such options.</div>
<div><br></div><div><br></div><div>I would be glad if mentors from xapian community can comment on my idea of implementing Language modelling technique and search result diversification as a project in scenario of <span style="font-size:13px;color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;line-height:18px">Open Source Search Engine Library(</span> xapian). Will implementing these techniques help xapian as a open source project?</div>
<div><br></div><div>wishing to join xapian community.</div><div><br></div><div><br></div>-- <br>with regards<br>Gaurav A.<br>
</div>