[Xapian-discuss] File Descriptors

[Richard Boulton]

On Fri, 2005-01-28 at 11:19 -0500, Mike Boone wrote:
> We are currently having problem with "too many open files" on our system. We
> have a busy server that runs Apache, MySQL, and PHP on Red Hat Enterprise AS
> 2.1. The most "exotic" thing the system runs is Xapian 0.8.3, which is
> accessed via the PHP-SWIG binding. I'm currently investigating the cause,
> and I thought I'd check with this list to see if there was anything obvious
> that had to do with Xapian.
>
> We're averaging something like 18,000 Xapian queries a day. Does each Xapian
> query create a lot of open files? Is there anything special I need to do to
> make sure those files get closed? Are there any special considerations since
> I'm using Xapian via the PHP-SWIG binding?

Each open Xapian database uses 5 file descriptors (and a 6th is used
whilst the database is being opened, or when the database is being
flushed).  This could lead to quite a large number of filedescriptors
being used.

On Linux I believe there are two file descriptor limits - one is a hard
limit set by the operating system (but is quite a high value).  The
other is a lower, per-process limit, which can be changed.  Under bash,
"ulimit -n" displays the current per-process limit, which may be
modified if you have sufficient permissions (usually root).

All file descriptors used by a Xapian database should be released when
the database is closed, so as long as there isn't a bug in PHP-SWIG
which is failing to delete databases after use, there shouldn't be a
problem that raising the limit won't solve.  I haven't looked at the PHP
bindings in detail, but it looks like you are meant to explicitly delete
database (and other Xapian objects) after use, so it would be worth
making sure that you're doing that.

-- 
Richard Boulton <richard at tartarus.org>