[Xapian-discuss] "Value in posting list too large" error with 1.1.4 (chert and brass, not flint)

Jason Tackaberry tack at urandom.ca
Sun Mar 7 20:58:37 GMT 2010


Hi,

I've a program which:

     1. Sets XAPIAN_FLUSH_THRESHOLD=1000
     2. Opens a (new) database for write
     3. Indexes a few thousand documents
     4. Periodically also does queries on the database

With 1.1.4, with certain document sets (basically a particular mail
folder of mine), Enquire.get_mset() sometimes (but not always) triggers
a "RangeError: Value in posting list too large" error.  Once this
happens, occasionally I also see DocNotFoundError.

After the indexing completes and a final explicit flush() is called and
the program exits, xapian-check reports no errors found, so at least the
database seems consistent on disk.

What's interesting is that if I set XAPIAN_FLUSH_THRESHOLD to be a value
larger than the document set size (and I am not explicitly flushing
either), the problem does not occur.

The problem occurs with the chert and brass backends, but not flint.
(Nor does it, perhaps obviously, occur with xapian-core 1.0.16.)  Just
to rule it out, I also tried 1.1.4 built with gcc 4.1 (rather than 4.2
with -fno-strict-aliasing) but the result was the same.

I am trying to reproduce the problem as a smaller test case, but haven't
been terribly successful so far.  I was hoping that someone might have
some ideas about possible explanations that might help me work up a
useful test case.

Thanks,
Jason.




More information about the Xapian-discuss mailing list