[Xapian-discuss] Xapian linked-in/port driver for Erlang?

[David P. Novakovic]

hmm, nice use case.. Interesting that the code gets CPU bound so badly. Have
you profiled the app?

My testing with high volume twisted xmlrpc servers (doing lots of small
requests) actually put the bottleneck at the xml parsing stage.. was a bit
of  surprise.

Anyway.. I'd personally be interested to see how you go with the erlang
driver.. I don't have the time to contribute at the moment unfortunately :(

David

On Fri, Nov 6, 2009 at 7:02 AM, Jarrod Roberson <jarrod at vertigrated.com>wrote:

> On Thu, Nov 5, 2009 at 4:54 AM, David P. Novakovic <
> davidnovakovic at gmail.com
> > wrote:
>
> > Unless you are sharding the xapian DB i'm not sure what advantage
> > erlang/OTP would give over a well written twisted frontend anyway... the
> > xapian library is not concurrent writesafe, so you might as well just run
> > multiple twisted nodes in front of it with a single process writing to
> it.
> > Obviously there is  ceiling on that too because of the IO restrictions..
> >
> > Perhaps our erlang friends have other suggestions though..
> >
> >
> the machine I am targeting this to is a 32 core machine, other servers we
> have written get CPU bound very quickly and I would rather run 1 Erlang VM
> vs 26+ Twisted instances, it is just way more manageable. This is to
> support
> a cluster of frontends to multiple Xapian databases eventually, my
> prototype
> in Twisted has gotten really complicated dealing with clusters and cache
> coherency and what not.
> Right now I am considering a Java/Terracotta implemenation and a Erlang
> implementation.
>
> --
> Jarrod Roberson
> 678.551.2852
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>