[Xapian-devel] Remote protocol abstracted

[Germán M. Bravo]

I have been working in the abstraction of the remote protocol to separate
it from the server and the connection.
The purpose of this is I need to use the Xapian remote protocol in a server
we are working on (named Xapiand:
https://github.com/Kronuz/Xapiand) which is similar to some extent to the
`xapian-tcpsrv` in that it also serves
databases remotely through TCP, using the binary protocol. What is
different about it is that it can serve any
number of databases using an event driven mechanism for receiving and
attending clients. Additionally, it also
serves clients using a custom HTTP protocol which is JSON based and RESTful.

I have the following commits in my Xapian fork at
https://github.com/Kronuz/xapian/commits/master:


1. Remote protocol with support for selection of database directory
-------------------------------------------------------------------

If the user requests to open a remote database server, it can optionally
require
the remote backend to tell the server what database it wants to open, if the
remote server is configured as a server without a default directory
(xapian-tcpsrv remains to always have a default directory at the moment).

The binary protocol was changed in the way that a new command can be sent to
the server (MSG_SELECT) in case the server tells the client no default
directory
is selected, the server then reacts to this command by opening and
selecting the
requested database as the active database.


2. Split msg_query and msg_getmset
----------------------------------

Originally, `msg_query()` handled two commands by the client: `MSG_QUERY`
and
also `MSG_GETMSET` (which was marked as "used during a conversation"). This
changes this and makes it so `msg_query()` returns even when the client
hasn't
yet sent a `MSG_GETMSET`, but instead only keeping the current satate of the
query in the `MatchState` struct and setting the expected `required_type`
for
the next command.

The purpose of this is that an event based remote protocol server might
still
not yet have the `MSG_GETMSET` message from the client as a response, and
it'd
be bad practice (or catastrophic) to block the thread while the message
arrives.
This blocking (if any) should be handled further up, by the event loop.


3. Method run() split in two: run() and run_one()
-------------------------------------------------

The method run() calls run_one() in a loop, but we don't always want this
behavior. The reason is, again, event based messaging might decide to start
attending other clients on the same thread without blocking, thus an
infinite
while for these type of servers is not desired.


4. Abstract RemoteProtocol class added
--------------------------------------
RemoteServer extends from abstract RemoteProtocol, and only implements a few
members needed by the protocol. This effectively separates the protocol
from the
connection and the server.

RemoteProtocol with virtual abstract methods for:

* get_message - Receives a new message (waiting when needed).
* send_message - Sends a message to the client.
* get_db - Reserves a database when the class needs to use it.
* release_db - Releases the database when the class is not using it.
* select_db - It's called when the protocol needs to select a new database
to work on.
* shutdown - It's called when the protocol requires to shutdown.

Each command in the RemoteProtocol now do a two additional things:

1. Use get_db() to reserve and get a read-only or writable database.
2. Relese the reserved database using release_db()

Implements RemoteProtocol on the RemoteServer:

* get_db - Always returns the same database object (db or wdb)
* release_db - Does nothing in this server (database is never released).
* get_message and send_message - Call RemoteConnection methods.


5. RemoteProtocol implementation moved to remoteprotocol.cc
-----------------------------------------------------------

Added `remoteprotocol.cc` with the implementation of `RemoteProtocol` and
header
file remoteprotocol.h with added `RemoteProtocol` abstract class definition
moved to `include/xapian/remoteprotocol.h`

The reason for this is other projects might want to use the `RemoteProtocol`
class for implementing a custom remote server. Xapiand is one of these
projects.
(https://github.com/Kronuz/Xapiand)


Please, any comments and feedback would be appreciated.

German M. Bravo (Kronuz)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20150401/3386c9f4/attachment.html>