[Xapian-tickets] [Xapian] #250: replace_document should make minimal changes to database file

[Xapian]

#250: replace_document should make minimal changes to database file
---------------------------+------------------------------------------------
 Reporter:  richard        |       Owner:  richard  
     Type:  enhancement    |      Status:  assigned 
 Priority:  normal         |   Milestone:  1.2.x    
Component:  Backend-Chert  |     Version:  SVN trunk
 Severity:  normal         |    Keywords:           
Blockedby:                 |    Platform:  All      
 Blocking:                 |  
---------------------------+------------------------------------------------

Comment(by cworth):

 I haven't studied what any of these patches do in detail. And I only
 know a little about what position data will be in a notmuch database.

 Most of the terms in the database are the content of the mail and for
 these we use Xapian::TermGenerator::index_text. That generates
 positions right? If if didn't we couldn't do phrase searches, right?
 (And we definitely can.)

 I carefully tested the master branch of Xapian (svn commit 13730),
 then with the lazy_positions.patch on top and then with
 patch_kan_ru_2.patch instead. Here are the results I got for an
 identical "notmuch tag" operations. Each operation here does a
 read/modify/write of 8933 documents, and the modification is the
 addition or removal of one or zero terms (most commonly one, not
 zero).

 I flushed the Linux page cache (echo 3 > /proc/sys/vm/drop_caches)
 between the different Xapian versions being tested here, (but not
 between the +foo and -foo commands for each).

 Xapian master
 -------------
 $ time ./notmuch tag +foo tag:sent
 real    3m26.163s
 user    2m48.031s
 sys     0m5.136s

 $ time ./notmuch tag -foo tag:sent
 real    3m11.516s
 user    2m42.254s
 sys     0m4.304s

 With lazy_positions.patch
 -------------------------
 $ time ./notmuch tag +foo tag:sent
 real    3m22.187s
 user    2m44.450s
 sys     0m4.864s

 $ time ./notmuch tag -foo tag:sent
 real    2m56.191s
 user    2m36.234s
 sys     0m3.808s

 With patch_kan_ru_2.patch
 -------------------------
 $ time ./notmuch tag +foo tag:sent
 real    0m43.120s
 user    0m34.206s
 sys     0m1.068s

 $ time ./notmuch tag -foo tag:sent
 real    0m29.294s
 user    0m27.790s
 sys     0m0.392s

 Perhaps the above is helpful.

 -Carl

-- 
Ticket URL: <http://trac.xapian.org/ticket/250#comment:13>
Xapian <http://xapian.org/>
Xapian