[Xapian-discuss] making my db leaner and meaner

[James Aylett]

On Fri, Mar 27, 2009 at 09:47:52AM +0000, Ben Campbell wrote:

> It's a Linux box (debian Lenny). We are getting kernel errors to do with 
> excess time taken to write to disk, exacerbated at least by my website - 
> iotop shows a large amount of time is spent by Apache loading (parts of) 
> the xapian index into RAM.

If the total apache core memory usasge (RSS) is topping out at close
to your total memory, then adding memory will help here. However if
you're not seeing this, it's possible that you're trying to drive your
disks harder than they can manage, in which case moving to RAID over
multiple spindles will probably help more.

Adding more memory is almost certainly easier, and is rarely a bad
thing unless it's hugely expensive. That plus slimming your database
will hopefully give a measurable improvement, but if it doesn't, try
looking into the disk usage more carefully. (For instance, iostat and
vmstat will give you an idea of how much the disks and partitions are
being used, and also how much page swapping is going on. It's
difficult to describe quite how to parlay that into an impression of
what's going on, but there are some good books and websites, and one
of the best ways of learning is to monitor lots and read up on what
all the figures mean!)

J

-- 
  James Aylett

  talktorex.co.uk - xapian.org - uncertaintydivision.org