[Xapian-tickets] [Xapian] #266: Add more control over the automatic flush and commit behaviour (was: Add a minimise_memory() method.)

[Xapian]

#266: Add more control over the automatic flush and commit behaviour
-------------------------+--------------------------------------------------
 Reporter:  richard      |       Owner:  olly     
     Type:  enhancement  |      Status:  assigned 
 Priority:  normal       |   Milestone:  1.3.0    
Component:  Library API  |     Version:  SVN trunk
 Severity:  normal       |    Keywords:           
Blockedby:               |    Platform:  All      
 Blocking:               |  
-------------------------+--------------------------------------------------

Comment(by richard):

 After further discussion on IRC, there seems to be agreement that making a
 special "non-sticky" behaviour for 0 would be surprising, and might not be
 a good idea.  Olly suggests that disabling implicit flushing is a bad
 idea, since certain operations cause implicit flushing anyway, and other
 tables still write directly to disk, so it doesn't control IO.  I'm not
 convinced this is a real problem, as long as these limitations are
 documented.

 I think there's a strong case for separating the commit and flush
 threshold - in RestPose, I currently force a commit after 5 seconds of
 inactivity, and otherwise commit using the automatic threshold.  I'm quite
 happy to leave the flush threshold at an value which causes an automatic
 commit (though it is certainly good to be able to control that using a
 method, rather than having to call putenv, and then open the database
 again).   However, when doing a batch update, I'd like to be able to avoid
 the overhead of the fsync() calls associated with commit until after the
 end of the batch update, so I would either set the threshold to a huge
 value, or to 0 if that disables automatic commits.

 I've put together a patch which implements a set_flush_threshold() and
 set_commit_threshold() method, but again run out of time to properly test
 it.  There's considerably more to test this time, too.  The patch adds a
 separate counter for the number of changes since the last flush and since
 the last commit (replacing the single changes_count which previously
 counted both).

 My main concern with the interface as implemented in the patch is how we
 handle units. While we could add an optional second parameter to specify
 units, there's no way to return the previous value of those units.  We can
 still make the set_flush_threshold(set_flush_threshold(0)) idiom work
 sensibly if the unit-less version of the call leaves the units unchanged.

-- 
Ticket URL: <http://trac.xapian.org/ticket/266#comment:11>
Xapian <http://xapian.org/>
Xapian