[Xapian-discuss] Suitability of Xapian for my application?

[Olly Betts]

On Thu, Oct 14, 2004 at 10:24:37PM -0700, Eric Parusel wrote:
> Xapian will reasonable be able to handle a corpus of let's say triple 
> that, 600K documents or more?

Yes.  The largest I've personally worked with is 18 million documents,
but I think people have done bigger systems.  Webtop peaked at around
500 million, but used the old muscat 3.6 backend rather than quartz so
it's hard to compare directly.  But I think quartz now comfortably
surpasses muscat 3.6 (equivalent quartz databases are smaller so less
I/O should be needed).

> How much RAM would Xapian take up while adding keywords, or searching 
> typically?

It depends what you set the autoflush threshold to, but for 50000 I get
around 250MB process size.  You need more RAM than that as you want the
OS to cache database blocks.

Cheers,
    Olly