[Xapian-tickets] [Xapian] #326: Searches with chert are slow, due to slow doclen access

Xapian nobody at xapian.org
Thu Feb 5 10:17:25 GMT 2009


#326: Searches with chert are slow, due to slow doclen access
---------------------------+------------------------------------------------
 Reporter:  richard        |        Owner:  richard  
     Type:  defect         |       Status:  assigned 
 Priority:  normal         |    Milestone:  1.1.0    
Component:  Backend-Chert  |      Version:  SVN trunk
 Severity:  normal         |   Resolution:           
 Keywords:                 |    Blockedby:           
 Platform:  All            |     Blocking:           
---------------------------+------------------------------------------------
Changes (by richard):

  * status:  new => assigned


Comment:

 http://tartarus.org/richard/xapian_perf_analysis/callgrind.out.xapian_search_chert_1
 is a callgrind trace for 1000 searches with chert, unpatched.

 My first attempt to improve performance was to reduce the size of the
 doclen postlist chunks, to avoid so much time being spent calling next.
 Since around 30 calls to next() were made per search, I reduced the
 chunksize for doclen lists from 2000 bytes to 70 bytes.  After applying
 xapian-compact to the original and new databases, the new database was a
 fraction of a percent larger (postlist.DB of 245088256 bytes instead of
 245022720 bytes). but searches have sped up - 10,000 searches completed in
 10.53 seconds instead of 12.58 seconds with the standard doclen chunk
 size.

 Profiling with callgrind (see
 http://tartarus.org/richard/xapian_perf_analysis/callgrind.out.xapian_search_chert_small_doclen_chunks
 )
 shows that move_forwards_in_chunk_to_at_least() is now taking only 23.75%
 of the CPU time (though this is still undesirably high), but
 ChertPostList::move_to_chunk_containing takes 50.51% of the CPU time.

-- 
Ticket URL: <http://trac.xapian.org/ticket/326#comment:1>
Xapian <http://xapian.org/>
Xapian



More information about the Xapian-tickets mailing list