[Xapian-tickets] [Xapian] #259: DatabaseError when interleaving reads and writes
Xapian
nobody at xapian.org
Mon Apr 28 10:14:02 BST 2008
#259: DatabaseError when interleaving reads and writes
---------------------------+------------------------------------------------
Reporter: richard | Owner: richard
Type: defect | Status: new
Priority: high | Milestone: 1.0.7
Component: Backend-Flint | Version: SVN HEAD
Severity: normal | Keywords:
Blockedby: | Platform: All
Blocking: |
---------------------------+------------------------------------------------
Alessandro Pasotti reported a problem to xapian-discuss on 23 April 2008,
where some PHP code, using the clustering branch of xapian, was performing
an indexing run and then a "cluster" operation on the same
WritableDatabase object without an intermediate flush. The cluster
operation returned DatabaseError: inflate failed (invalid distance too far
back). He later sent me a testcase by private email which I was able to
use to track down the problem, and work out a smaller testcase.
I was quickly able to reproduce the problem on SVN HEAD, and have now
worked out a minimal testcase which reproduces the same problem, in a
different way.
I have narrowed down the problem to the "next_for_sequential" method in
FlintTable. This is called (via next()) by FlintTable.read_tag(), when
reading the subsequent components of a multipiece tag, and should never
return false. (We should probably have a check in FlintTable.read_tag()
that next() doesn't return false, which raises DatabaseCorruptError if it
does).
However, when the next piece of a tag is in a later block than the current
block, and that new block has been newly added (since the last flush, or
the database was opened), next_for_sequential incorrectly returns false.
If I comment out the call to next_for_sequential in next(), so that
next_default() is always used instead, the code works correctly.
I believe the problem is that next_for_sequential checks whether a block
is in the database by comparing its number with
base.get_last_block(). However, the last_block member in base is only
updated when a flush is called, so doesn't reflect newly added blocks.
Thush, next_for_sequential returns false incorrectly.
This bug affects both Flint and Chert databases.
--
Ticket URL: <http://trac.xapian.org/ticket/259>
Xapian <http://trac.xapian.org>
Xapian
More information about the Xapian-tickets
mailing list