[Xapian-tickets] [Xapian] #326: Searches with chert are slow, due to slow doclen access
Xapian
nobody at xapian.org
Thu Feb 5 10:17:25 GMT 2009
#326: Searches with chert are slow, due to slow doclen access
---------------------------+------------------------------------------------
Reporter: richard | Owner: richard
Type: defect | Status: assigned
Priority: normal | Milestone: 1.1.0
Component: Backend-Chert | Version: SVN trunk
Severity: normal | Resolution:
Keywords: | Blockedby:
Platform: All | Blocking:
---------------------------+------------------------------------------------
Changes (by richard):
* status: new => assigned
Comment:
http://tartarus.org/richard/xapian_perf_analysis/callgrind.out.xapian_search_chert_1
is a callgrind trace for 1000 searches with chert, unpatched.
My first attempt to improve performance was to reduce the size of the
doclen postlist chunks, to avoid so much time being spent calling next.
Since around 30 calls to next() were made per search, I reduced the
chunksize for doclen lists from 2000 bytes to 70 bytes. After applying
xapian-compact to the original and new databases, the new database was a
fraction of a percent larger (postlist.DB of 245088256 bytes instead of
245022720 bytes). but searches have sped up - 10,000 searches completed in
10.53 seconds instead of 12.58 seconds with the standard doclen chunk
size.
Profiling with callgrind (see
http://tartarus.org/richard/xapian_perf_analysis/callgrind.out.xapian_search_chert_small_doclen_chunks
)
shows that move_forwards_in_chunk_to_at_least() is now taking only 23.75%
of the CPU time (though this is still undesirably high), but
ChertPostList::move_to_chunk_containing takes 50.51% of the CPU time.
--
Ticket URL: <http://trac.xapian.org/ticket/326#comment:1>
Xapian <http://xapian.org/>
Xapian
More information about the Xapian-tickets
mailing list