[Xapian-tickets] [Xapian] #326: Searches with chert are slow, due to slow doclen access

[Xapian]

#326: Searches with chert are slow, due to slow doclen access
---------------------------+------------------------------------------------
 Reporter:  richard        |       Owner:  olly     
     Type:  defect         |      Status:  assigned 
 Priority:  highest        |   Milestone:  1.1.4    
Component:  Backend-Chert  |     Version:  SVN trunk
 Severity:  normal         |    Keywords:           
Blockedby:                 |    Platform:  All      
 Blocking:                 |  
---------------------------+------------------------------------------------
Changes (by olly):

  * status:  new => assigned
  * milestone:  1.1.3 => 1.1.4


Comment:

 I've committed more efficient pack and unpack functions for chert to trunk
 in r13578, which should improve the situation here.  These sped up
 searching for "The" on gmane by about 35%.  If that same improvement were
 seen in the original testcase, chert would be 4.6 times slower rather than
 7 times, which is a good start.

 This patch incorporates the ideas from avoid_string_appends_1.patch.  I've
 not used the unrolled/specialised versions (yet anyway) as I'm worried
 that inlining these will cause excessive code size growth, filling the CPU
 cache more.  So we need to re-benchmark them in a real situation, not just
 a microbenchmark.  I did address the string::insert() issue which is one
 issue that the unrolling in avoid_string_appends_and_loop_unroll_2.patch
 is aimed at.  Also, unpack_uint() now finds the length first (which allows
 a cleaner unpacking loop working backwards), so the trick of checking
 there are at least 5 bytes available and using a special unrolled version
 is likely to be less of a gain.

 Interestingly, for the gmane search for "The", memcpy() dominates the
 callgrind profile (both before and after).

 Bumping remaining work to 1.1.4.

-- 
Ticket URL: <http://trac.xapian.org/ticket/326#comment:16>
Xapian <http://xapian.org/>
Xapian