[Xapian-tickets] [Xapian] #326: Searches with chert are slow, due to slow doclen access

[Xapian]

#326: Searches with chert are slow, due to slow doclen access
---------------------------+------------------------------------------------
 Reporter:  richard        |        Owner:  richard  
     Type:  defect         |       Status:  assigned 
 Priority:  normal         |    Milestone:  1.1.0    
Component:  Backend-Chert  |      Version:  SVN trunk
 Severity:  normal         |   Resolution:           
 Keywords:                 |    Blockedby:           
 Platform:  All            |     Blocking:           
---------------------------+------------------------------------------------

Comment(by richard):

 Applying avoid_string_operations.patch and optimise_unpack_uint.patch
 together, I get a search time of 7.84 seconds with the full size doclen
 chunks, or 7.70 seconds with the small doclen chunks.  I think this is
 getting towards the limit of what can be acheived with this kind of
 optimisation, and a better datastructure for the doclens which can find
 document lengths without linear seeking is needed.

 Looking at the callgrind output for this (with the full size doclen
 chunks), ChertPostList::move_to_chunk_containing() is currently using 221
 757 608 cycles, whereas with flint, the entire set of calls to next() uses
 only 214 203 320 cycles.  This implies that to get faster performance than
 flint with chert, we'll need to decrease the time spent in
 move_to_chunk_containing() - one way to do this would be to decrease the
 number of calls, by reducing the number of chunks (either by making them
 larger, or more compact).

-- 
Ticket URL: <http://trac.xapian.org/ticket/326#comment:5>
Xapian <http://xapian.org/>
Xapian