[Xapian-discuss] Xapian and quartz scalability - feedback of current users

[Alexandre Dulaunoy]

Hi All,

We would like to make some test with Xapian and the quartz backend on
a large set of sample test document (around 50 millions for starting)
. quartz backend seems very flexible and the document scalability on
the web site (http://www.xapian.org/docs/scalability.html) is talking
of a possible way to implement a kind of cluster for concurrent search
indexing and asynchronous updating. We were wondering if there is any
users of quartz usting a clustering approach in the list. What is(are)
the classical design ? Based on what the separation of the quartz
databases is made ? How is the updating handled to provide continuous
services ? How is the cluster organized ? How are you dealing with
unresponsive systems part of the cluster ? Is there any other free
software components available for job allocation (updating, compacting
and alike)  inside a quartz/xapian cluster ? Is there any technical
comparison between Nutch/Lucene and Xapian/Quartz regarding large
scale index ?

Thanks a lot for any feedback,

adulau