[Xapian-discuss] Re: Remote databases and daemons

[Michel Pelletier]

Philip Neustrom wrote:

> Here's what I'm thinking:  Write a small xapian-daemon server in
> Python that listens on TCP and can index and search.  Because xapian
> can only do one write at a time (last i checked?) the server will keep
> a queue of index requests and apply them in order/thread to avoid
> blocking.  Is this something that's useful or is there a more
> xapiantic way to do this?

Here is a twisted perspective broker server object that is xapwrap 
specific but gives you the basic pattern for a multi-threaded reader 
daemon in Python.  Just replace the xapwrap read only indexes with 
xapian databases, etc.  xaql.query_xapwrap is a function that does the 
actual query which you would also replace with your own.

As for writing a single thread and connection is all that's needed and 
obvious enough.

#! /usr/bin/python

from twisted.internet import threads
from twisted.spread import pb
from twisted.python import threadable
threadable.init()

from xapwrap.index import SmartIndex, SmartReadOnlyIndex
from xapwrap.document import Document, TextField, SortKey

import pool
import xaql

class Xapd(pb.Root):

     def __init__(self, path):
         self.readers = pool.Pool(pool.Constructor(SmartReadOnlyIndex, 
path))

     def remote_query(self, query):
         d = self.readers.get()
         r = threads.deferToThread(xaql.query_xapwrap, d, query)
         self.readers.put(d)
         return r

if __name__ == '__main__':

     from twisted.internet import reactor
     reactor.suggestThreadPoolSize(10)
     reactor.listenTCP(3333, 
pb.PBServerFactory(Xapd('/home/michel/tmp/index5')))
     reactor.run()