[Xapian-discuss] Benchmark (index, search) on 20 million docs Xapian/Linux vs. Lucene/Windows

[Charlie Hull]

James Aylett wrote:
> On Tue, Feb 13, 2007 at 09:57:32AM +0000, Olly Betts wrote:
> 
>>>> Result: Xapian/Linux showed to be 35% faster
>>>> then Lucene/Windows on searching.
>>> That's the typical Windows overhead for finding and opening files on 
>>> windows compared to linux.
>> While I can see indexing may involve opening a lot of files, searching a
>> database opens relatively few.  It's about 11 for Xapian.  I don't know
>> how many it would be for Lucene, but it seems unlikely it's many more,
>> so you'd need a really large overhead per file find/open to incur a 35%
>> overhead over the total time taken to search a database of this size.
> 
> Nonetheless, it would be interesting to see Lucene/Linux on the same
> hardware and OS install, for comparison. It may make a difference on
> Windows which install of the OS you're using, as (guessing here, but
> with I hope some justification) the disk optimisations around
> read-ahead may vary. There's a tunable somewhere that affects these
> things. Not sure if Lucene/Windows counts as a foreground or
> background app in terms of the performance boost tunable, either (or
> what that was set to).

..Or indeed, Xapian on Windows. We know of some performance tests being 
done on this platform and we are planning some of our own (we'll feed 
back to the list). Performance figures are very valuable when trying to 
convince people to use Xapian.

I'm working with Richard this week on getting Xapian compiled using the 
free Visual C++ 2005 Express Edition, so it should be relatively easy to 
get native binaries for Windows - or of course there's always Cygwin.

Charlie