[Xapian-devel] multiple writers and remotetcp backends

[Mark Hammond]

Hi guys,
  We've been in discussion with Richard and Olly on this issue, in various
different forums, but as the correct answer isn't immediately obvious, I'm
opening it up for wider discussion and comment.

The problem is that a xapian tcp-server in 'writable' mode makes no attempt
to ensure only one 'active' connection at a time is trying to modify the
database.  If multiple connections are made to a writable server, the
behaviour is undefined (or even it is was defined, it is unlikely to be
defined in a way that would make it useful).  While some applications can
ensure internally that only a single connection is made to such a server,
some applications are architected such that multiple processes, possibly
even multiple machines, must coordinate this "single writer" approach.  This
becomes quite difficult without support inside xapian itself.

It seems there are 2 general solutions we can implement:

* Only ever allow a single connection to the writable server.  When a second
connection is attempted, we either refuse the connection, or allow the
connection just to send back an authorative 'writer already connected'
response, and then close the connection.

* Implement a kind of 'queue' or some other way to block the incoming
connection.  In this case we would accept the connection, respond with a
message indicating they are in a queue (your call is important!) and then
block until the first writer is complete.  The client side of the connection
then has a choice regarding waiting in the queue, or hanging up and trying
again later.


In my opinion, the second option sounds the "best", but the first option
seems "good enough" and easier to implement.  I'm sure there are both other
options and opinions on this topic, so I'm soliciting all feedback on this,
with the intention of opening a xapian bug to track the status, and
ultimately end up with a patch.

Thanks,

Mark