[Xapian-discuss] Working Demo for WWW Search Engine

[Olly Betts]

On Wed, Mar 01, 2006 at 09:59:13AM -0800, Kevin SoftDev wrote:
> Running on the Suse 10.0 Pentium 2.8 GHz with 2 GB memory the search started
> to slow down using MySQL 5.0 where some results were coming after 10-15
> seconds and the CPU usage was approaching 99% and memory usage 25%.

I've heard other reports that Xapian beats MySQL's free text search
pretty resoundingly on large datasets.  It's not too suprising really -
MySQL's focus is as an RDBMS, whereas Xapian's is searching free text.

I am suprised they manage to be CPU bound on a large dataset though.

> With Xapian I see CPU usage between 3-4% per search and memory usage only
> 0.3%.

I suspect that 0.3% probably doesn't include all the disk blocks that
the Linux kernel has cached from the Xapian database, so if you tried to
run the Xapian search on a machine with 7MB of spare RAM (i.e. 0.3% of
2GB rounded up) you'd find it ran rather slowly.  But then MySQL is
presumably also benefitting from such caching too.

Cheers,
    Olly