[Xapian-devel] question and potential consulting gig

[Olly Betts]

On Tue, Jul 28, 2009 at 12:45:33PM -0500, Kyle J. Ginavan wrote:
> we are looking to enhance our projects search.  we only need searching  
> of database tables.  we are currently using postgresql's full text  
> search.
> 
> from your experience, how long does xapian index take to add delta's  
> to results?  on your site you advertise "Allows simultaneous update  
> and searching. New documents become searchable right away."  what does  
> right away mean?  like less than a minute, 5 minutes (given unlimited  
> hardware resources)?  is there db size where adding delta's is slow?

It going to be rather dependent on the data and the hardware, but to
give you an idea, you can add a single document to a database of 600,000
wikipedia articles in 6 seconds elapsed time.

Batching is much more efficient than adding single documents - for the
same database, you can add a batch of 24 documents in 27 seconds.

If you really have unlimited hardware resources, you should be able to
do much better (e.g. using SSDs instead of hard disks).  The main
factors here are going to be the speed of the disk subsystem and having
enough RAM to keep the parts of the Xapian database being touched in
the VM cache.

> if someone can confirm this, i would like to discuss a potential  
> consulting job with them.

Feel free to email me off-list to discuss.

Cheers,
    Olly