[Xapian-devel] Database Locking

Jeff Stearns jeff.stearns+gmane at gmail.com
Sat Jan 23 02:27:42 GMT 2010


Olly Betts <olly <at> survex.com> writes:
 
> Locking of Quartz databases is currently done with a lockfile.  This
> works well from the actual locking side, but the downside is that
> the lock isn't released if a process doesn't destroy the
> Xapian::WritableDatabase object.  This is made worse because some
> of the bindings don't call destructors (or don't do it reliably).
> 
> The obvious alternative is to use actual locking APIs.  On Unix, this
> mean fcntl.  Windows has a suitable API too.
> 
> But the problem is that on Unix, fcntl locks are per process.  So if
> we used fcntl locking, then if a process tried to open the same database
> twice as a WritableDatabase, it would succeed.  This is probably more of
> a worry in a threaded application, but threads aren't actually needed to
> hit this problem.

Olly -

I know nothing about the internals of Xapian, but I wonder whether this
cunning plan is more complex  and expensive than necessary.

I wonder why Xapian doesn't apply the flock, then set a flag indicating that
the database is locked.  Now whenever Xapian goes to open a database,
it would first check whether the flag is set.  If so, Xapian knows that that
the database is already open within this process.  If the flag is not set,
Xapian continues onward, probing the file with flock to see whether it's
open within some other process.

This should work in a threaded environment so long as the customary
synchronization primitives are used to avoid race conditions.
 
> So that's why we've stuck with lock files so far.  But I have a cunning
> plan...
> 
> We fork() and then fcntl() lock from a different process.  Since each
> WritableDatabase now has its own locking process, fcntl() locking works!
> 
> I wondered about fork() fcntl() exec("/bin/true") and then just kill the
> child process to release the lock.  But we want to be sure that the
> child process dies when the parent does, so it might be better to have
> our own little helper program.  We could open a pipe to the helper,
> which then blocks on read or select until the other end of the pipe is
> closed either explicitly or by the main process terminating.  Hmm, that
> sounds a lot like "/bin/cat" actually!




More information about the Xapian-devel mailing list