[Xapian-discuss] xapian-tcpsrv php in real fastcgi loop

[Olly Betts]

On Sun, Mar 21, 2010 at 11:57:03AM +0100, Martin Regner wrote:
> Now i randomly get an DatabaseLockError when one of the loops is trying to
> index something (on the writing instance). I try to solve this by
> implementing a few retries after some microseconds. But this is like betting
> on a lucky chance. Is this a problem inside the tcpsrv or is it (like before)
> the not yet realy closed client connection of the loop/thread?

It's hard to be sure just from the description, but I suspect it is the
delay between the client closing the connection and the server noticing and
closing the WritableDatabase object, releasing the write lock.

In 1.1.x, you could explicitly call close() on the remote database in the
client.  I'm not sure I see a better fix than retrying for 1.0.x.  An
explicit call to flush() before closing will help, and is good practice as
it means you actually see any errors in committing the batched changes.

> Doeas anyone have another approach for multiple clients trying to write 
> simultanously?

Serialising all writes through a single writer is the usual approach to this.
If you put updates into some sort of queueing system, you also get the
ability to shut off the application of updates temporarily which can be handy.

> May it's possible that xapian-tcpsrv is buffering the write requests until
> the lock is removed (i know this is much more complicated than it sounds).

I'm not quite sure I follow.  If you're suggesting an explanation of the
issue, then the OS will buffer writes and then calls to fdatasync() will
mean we wait for them to hit disk before the lock gets released.

If you're suggesting a solution, I don't understand exactly what you are
proposing.

Cheers,
    Olly