[Xapian-devel] Database Locking

[Kevin Duraj]

We should try to avoid to do anykind of locking because it affects  
performance for everybody. Search engine is not database where user is  
allow to lock tables and make updates.

You can easily implement your own locking localy on your machine. To  
the rest of the users who are using Xapian index read only mode would  
be great overhead, and slow down in performance.

- Kevin Duraj

On Jan 22, 2010, at 6:27 PM, Jeff Stearns <jeff.stearns 
+gmane at gmail.com> wrote:

> Olly Betts <olly <at> survex.com> writes:
>
>> Locking of Quartz databases is currently done with a lockfile.  This
>> works well from the actual locking side, but the downside is that
>> the lock isn't released if a process doesn't destroy the
>> Xapian::WritableDatabase object.  This is made worse because some
>> of the bindings don't call destructors (or don't do it reliably).
>>
>> The obvious alternative is to use actual locking APIs.  On Unix, this
>> mean fcntl.  Windows has a suitable API too.
>>
>> But the problem is that on Unix, fcntl locks are per process.  So if
>> we used fcntl locking, then if a process tried to open the same  
>> database
>> twice as a WritableDatabase, it would succeed.  This is probably  
>> more of
>> a worry in a threaded application, but threads aren't actually  
>> needed to
>> hit this problem.
>
> Olly -
>
> I know nothing about the internals of Xapian, but I wonder whether  
> this
> cunning plan is more complex  and expensive than necessary.
>
> I wonder why Xapian doesn't apply the flock, then set a flag  
> indicating that
> the database is locked.  Now whenever Xapian goes to open a database,
> it would first check whether the flag is set.  If so, Xapian knows  
> that that
> the database is already open within this process.  If the flag is  
> not set,
> Xapian continues onward, probing the file with flock to see whether  
> it's
> open within some other process.
>
> This should work in a threaded environment so long as the customary
> synchronization primitives are used to avoid race conditions.
>
>> So that's why we've stuck with lock files so far.  But I have a  
>> cunning
>> plan...
>>
>> We fork() and then fcntl() lock from a different process.  Since each
>> WritableDatabase now has its own locking process, fcntl() locking  
>> works!
>>
>> I wondered about fork() fcntl() exec("/bin/true") and then just  
>> kill the
>> child process to release the lock.  But we want to be sure that the
>> child process dies when the parent does, so it might be better to  
>> have
>> our own little helper program.  We could open a pipe to the helper,
>> which then blocks on read or select until the other end of the pipe  
>> is
>> closed either explicitly or by the main process terminating.  Hmm,  
>> that
>> sounds a lot like "/bin/cat" actually!
>
>
> _______________________________________________
> Xapian-devel mailing list
> Xapian-devel at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-devel