[Xapian-tickets] [Xapian] #259: DatabaseError when interleaving reads and writes

Xapian nobody at xapian.org
Mon Apr 28 10:14:02 BST 2008


#259: DatabaseError when interleaving reads and writes
---------------------------+------------------------------------------------
 Reporter:  richard        |       Owner:  richard 
     Type:  defect         |      Status:  new     
 Priority:  high           |   Milestone:  1.0.7   
Component:  Backend-Flint  |     Version:  SVN HEAD
 Severity:  normal         |    Keywords:          
Blockedby:                 |    Platform:  All     
 Blocking:                 |  
---------------------------+------------------------------------------------
 Alessandro Pasotti reported a problem to xapian-discuss on 23 April 2008,
 where some PHP code, using the clustering branch of xapian, was performing
 an indexing run and then a "cluster" operation on the same
 WritableDatabase object without an intermediate flush.  The cluster
 operation returned DatabaseError: inflate failed (invalid distance too far
 back).  He later sent me a testcase by private email which I was able to
 use to track down the problem, and work out a smaller testcase.

 I was quickly able to reproduce the problem on SVN HEAD, and have now
 worked out a minimal testcase which reproduces the same problem, in a
 different way.

 I have narrowed down the problem to the "next_for_sequential" method in
 FlintTable.  This is called (via next()) by FlintTable.read_tag(), when
 reading the subsequent components of a multipiece tag, and should never
 return false.  (We should probably have a check in FlintTable.read_tag()
 that next() doesn't return false, which raises DatabaseCorruptError if it
 does).

 However, when the next piece of a tag is in a later block than the current
 block, and that new block has been newly added (since the last flush, or
 the database was opened), next_for_sequential incorrectly returns false.
 If I comment out the call to next_for_sequential in next(), so that
 next_default() is always used instead, the code works correctly.

 I believe the problem is that next_for_sequential checks whether a block
 is in the database by comparing its number with
 base.get_last_block().  However, the last_block member in base is only
 updated when a flush is called, so doesn't reflect newly added blocks.
 Thush, next_for_sequential returns false incorrectly.

 This bug affects both Flint and Chert databases.

-- 
Ticket URL: <http://trac.xapian.org/ticket/259>
Xapian <http://trac.xapian.org>
Xapian



More information about the Xapian-tickets mailing list