[Xapian-discuss] Filesystems

[Arjen van der Meijden]

Apart from noatime we haven't used any specific additional mount- or 
mkfs options yet. But we're going to check a few to see how they'll do.

Best regards,

Arjen

On 1-7-2009 12:00, Frank John Bruzzaniti wrote:
> Arjen,
> 
> Are you using any mount options optimisations like noatime or noboundary 
> with xfs?
> Are you using any mount option optimisations with ext4?
> 
> 
> Arjen van der Meijden wrote:
>> My colleague is testing several filesystems on our new search-machine. 
>> He has been looking at a few of the filesystems available in the 
>> 2.6.30 linux kernel, ext2/3/4, xfs, btrfs, nilfs2 and reiser4.
>>
>> "Unfortunately" the new machine has 24GB ram and 4x ssd in raid5. To 
>> get somewhat IO-bound results we had to cripple the machine (by making 
>> sure it couldn't use 20gb of those 24gb for file-cache) *and* 
>> cherry-pick our queries (only the heaviest with phrase-queries and such).
>>
>> In the normal scenario of having the full ram (or even 4gb) available 
>> and the ssd's backing up any cache-miss, it is simply cpu-bound. And 
>> that is with the fastest x86 2-socket cpu's available right now, a 
>> pair of intel X5570's. The good news is that it actually appears to 
>> scale very well when using more cpu-cores (this one has 8 cores with 8 
>> hyper-threading cores) and that we can get about 90 searches per 
>> second out of it, which is more than we do now per minute (and we 
>> haven't benchmarked the compacted database yet).
>>
>> I.e. our results indicate that for our reads it hardly matters which 
>> filesystem to pick, most of the database will be in RAM any way.
>>
>> With the crippled, extra-io, read-scenario, we do see differences in 
>> performance between de filesystems tested.
>>
>> When finished, we'll have numbers for linear writes (copying the 25GB 
>> database from another disk array), non-linear writes (updating the 
>> database) and semi-linear writes (compacting the database) with 
>> semi-linear (memory-backed) reads.
>> And of course the numbers for the crippled read-scenario.
>>
>> So far ext4 and xfs seem to be the best choices, both in read and 
>> write scenario's. But obviously, our numbers are done with ssd, not 
>> normal disks.
>>
>> We haven't yet tested the various mount/mkfs-options (apart from 
>> enabling noatime), but we'll probably settle for ext4 and then try a 
>> few options to better suit the filesystem to the underlying 
>> blockdevice. After that we'll also try what the various 
>> compaction-options do to the read-performance.
>>
>> Best regards,
>>
>> Arjen
>>
>> On 1-7-2009 3:18 James Aylett wrote:
>>  
>>> On Tue, Jun 30, 2009 at 06:12:55PM -0700, Kevin Duraj wrote:
>>>
>>>    
>>>> Based on my observation Flint runs best on ext3 filesystems
>>>>       
>>> I don't suppose you're able to share any of your numbers from this? I
>>> know it's not always possible, but having something on the wiki would
>>> be useful to people, if only to point them in useful directions of how
>>> to construct their own testing. (I assume you were comparing against
>>> JFS and XFS, maybe Reiser?)
>>>
>>> J
>>>
>>>     
>>
>> _______________________________________________
>> Xapian-discuss mailing list
>> Xapian-discuss at lists.xapian.org
>> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>>   
>