[Xapian-discuss] Unexpected slowness

[Daniel Andersson]

On 8 Feb 2007, at 12:44, Arjen van der Meijden wrote:

> On 6-2-2007 16:40 Daniel Andersson wrote:
>> On 5 Feb 2007, at 20:17, Olly Betts wrote:
>> faster as in faster cpu, faster disks and faster memory
>> but it only has 1 gb of ram
>> it's running on megaraid with scsi-disks. from bonnie++:
>> Version  1.03       ------Sequential Output------ --Sequential  
>> Input- --Random-
>>                     -Per Chr- --Block-- -Rewrite- -Per Chr- -- 
>> Block-- --Seeks--
>> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % 
>> CP  /sec %CP
>> septum           2G 14061  54 14601   5  3258   0  5026  16   
>> 4289   0 219.7   0
>>                     ------Sequential Create------ --------Random  
>> Create--------
>>                     -Create-- --Read--- -Delete-- -Create-- -- 
>> Read--- -Delete--
>>               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec % 
>> CP  /sec %CP
>>                  16 12837  44 +++++ +++ 14069  57 14783  55 +++++ + 
>> ++ 12375  59
>> septum,2G, 
>> 14061,54,14601,5,3258,0,5026,16,4289,0,219.7,0,16,12837,44,+++++,++ 
>> +,14069,57,14783,55,+++++,+++,12375,59
>
> If I read that correctly, your only reading by about 4-5MB/sec from  
> that disk-array? That is not very fast... Last week I did a test  
> (also with bonnie++ 1.03) on an array of four 320GB sata disks in  
> raid5 on a LSI SATA-8X with 256MB ram (a sata-megaraid controller),  
> which does about 80MB/sec block reads and something like 40MB/sec  
> for char reads. The single 500GB ide-disk that is also in the same  
> box got to 40MB/sec and 30MB/sec respectively orso.
> So my single disk is already 8 times faster in reads and my simple  
> raid array 20 times, than your set-up. Writes are also faster on  
> both set-ups, but thats a bit skewed since IDE and SATA drives  
> generally lie about being finished with their write.
>
> Of course the results of bonnie can vary a bit, depending on  
> concurrent work load and the exact set-up, but your set-up doesn't  
> strike me as very fast. Perhaps you need to adjust the device's  
> caching strategies, but otherwise you may actually be better of  
> with a single modern drive, rather than your current array. Of  
> course with a relatively small database like your 1.7GB, you can  
> easily add more ram to get the best possible performance, since  
> that will always be faster than a fast disk set-up, once things are  
> cached.

very true
re-ran the test, twice

first rerun:
Version  1.03       ------Sequential Output------ --Sequential Input-  
--Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- -- 
Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % 
CP  /sec %CP
septum           2G 17131  66 17177   7  5477   1 19340  64 31909   4  
293.8   0
                     ------Sequential Create------ --------Random  
Create--------
                     -Create-- --Read--- -Delete-- -Create-- -- 
Read--- -Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec % 
CP  /sec %CP
                  16 28301  96 +++++ +++ 26426  98 28813  94 +++++ ++ 
+ 23745  94
septum,2G, 
17131,66,17177,7,5477,1,19340,64,31909,4,293.8,0,16,28301,96,+++++,++ 
+,26426,98,28813,94,+++++,+++,23745,94


second rerun:
Version  1.03       ------Sequential Output------ --Sequential Input-  
--Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- -- 
Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % 
CP  /sec %CP
septum           2G 17177  66 17653   7  5806   1 18528  61 28103   3  
299.2   0
                     ------Sequential Create------ --------Random  
Create--------
                     -Create-- --Read--- -Delete-- -Create-- -- 
Read--- -Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec % 
CP  /sec %CP
                  16 28960  97 +++++ +++ 25687 100 28535 101 +++++ ++ 
+ 20194  85
septum,2G, 
17177,66,17653,7,5806,1,18528,61,28103,3,299.2,0,16,28960,97,+++++,++ 
+,25687,100,28535,101,+++++,+++,20194,85

/ d