[Xapian-discuss] Re: Optimization and Load balancing with Xapian

[David Levy]

I've done some tests this morning and it seems that some of this slowlyness
is due to sorting.

Indeed, Omega requests with and with sorting do not produce the same
calculation time at all. < 1 s without sorting and sometimes > 30 s with
sorting.... These 30 seconds happen with results having like 500+ matches.
How can it be possible ? Sorting should not be so much time consuming I
guess. I think I am missing something in Omega cgi parameters, Omega
template, or scriptindex configuration.
Can you help please :) ??

For information, the field I use for sorting is a numeric field containing
integers between ~ 1 and 100.

Regards
David


On 2/15/06, David Levy <dvid.levy at gmail.com> wrote:
>
> Hi (that's me again !! sorry :))
>
> I am experiencing bad response times with Xapian/Omega in the last few
> days. My database has more than 700k records, using ~ 3Go disk space.
> Maybe my requests or my templates are not optimized, or maybe it's a
> hardware (disk speed) issue. The weird thing is that often, the search time
> provided in the response is sub second, and the response is actually given
> by Omega over one second (even seconds ...). Did one of you had this kind of
> issues ? Maybe my Apache (1.x) is not well configured for Omega ?
> FYI, I am using PHP5 (so bindings are not available) and calling Omega
> with HTTP GET requests returning XML documents between two servers.
>
> To solve this issue, I was been thinking about load balancing Xapian. I
> could not find any information about that on Internet. One of you did it yet
> ? How ?
> I've found Crossroads (
> http://public.e-tunity.com/crossroads/crossroads.html) tonight, i think it
> could help :
> "*Crossroads is a load balance and fail over utility for TCP based
> services. It is a daemon program running in user space, and features
> extensive configurability, polling of back ends using 'wakeup calls',
> detailed status reporting, 'hooks' for special actions when backend calls
> fail, and much more. Crossroads is service-independent: it is usable for
> HTTP(S), SSH, SMTP, DNS, etc."
>
> *Thanks in advance !
>
> --
> David LEVY {selenium}
> Website ~ http://www.davidlevy.org
> Wishlist Zlio ~ http://david.zlio.com/wishlist
> Blog ~ http://selenium.blogspot.com
>



--
David LEVY {selenium}
Website ~ http://www.davidlevy.org
Wishlist Zlio ~ http://david.zlio.com/wishlist
Blog ~ http://selenium.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.tartarus.org/pipermail/xapian-discuss/attachments/20060215/859be7ec/attachment.htm