[Xapian-discuss] DFS solution

Lucian Nicolescu lucian at tonicagency.com
Sun Sep 9 11:54:11 BST 2007


Hi all,

I have such a solution in mind to solve high traffic issues:
- set up xmlrpc servers that communicate with the xapian db and can be
queried from any programming language
- balance the xmlrpc servers using pen (http://siag.nu/pen/) - from what I
read and heard this might scale just fine (uses round robin to balance
requests and detects failed services from the pool automatically).

This might work,
Lucian Nicolescu  

> -----Original Message-----
> From: xapian-discuss-bounces at lists.xapian.org 
> [mailto:xapian-discuss-bounces at lists.xapian.org] On Behalf Of 
> Olly Betts
> Sent: Tuesday, September 04, 2007 3:38 AM
> To: xapian-discuss at lists.xapian.org
> Subject: Re: [Xapian-discuss] DFS solution
> 
> On Tue, Jul 10, 2007 at 09:56:36AM +0100, James Aylett wrote:
> > On Mon, Jul 09, 2007 at 12:20:08PM -0700, Andrey wrote:
> > 
> > > I am currently doing some research on running xapian on 
> Distributed
> > > File Sysyem(DFS), try to find some cheap solution to scale up the
> > > storage, and, without any luck...
> 
> I've not tried running Xapian on a distributed file system, so I can't
> really comment there.  I'd certainly be interested to hear any
> experiences you have.
> 
> If your aim is to scale to a lot of traffic rather than enormous
> databases, then simply replicating the search boxes and 
> having some sort
> of load balancing is probably the lowest overhead approach.  If you're
> replicating widely, you need to consider how the system will handle a
> box which crashes.
> 
> > In general though, providing a file system has normal unix 
> semantics,
> > Xapian should run over it.
> 
> One important requirement is that fcntl locking is supported. 
>  E.g. NFS
> without lockd doesn't work (you should get an error explaining this).
> 
> Cheers,
>     Olly
> 
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
> 




More information about the Xapian-discuss mailing list