[Xapian-tickets] [Xapian] #455: "Value in posting list too large" error with 1.1.4 (chert and brass, not flint)

[Xapian]

#455: "Value in posting list too large" error with 1.1.4 (chert and brass, not
flint)
---------------------------+------------------------------------------------
 Reporter:  tack           |       Owner:  olly 
     Type:  defect         |      Status:  new  
 Priority:  normal         |   Milestone:       
Component:  Backend-Chert  |     Version:  1.1.4
 Severity:  normal         |    Keywords:       
Blockedby:                 |    Platform:  All  
 Blocking:                 |  
---------------------------+------------------------------------------------
 I am able to trigger "Value in posting list too large" in
 Enquire.get_mset() with 1.1.4 using chert and brass (but not flint) on
 certain document sets (which I unfortunately cannot share) using the
 following general approach:

   1. Set XAPIAN_FLUSH_THRESHOLD=1000
   2. Open a (new) database for write
   3. Index a few thousand documents
   4. Periodically query the database

 If I set XAPIAN_FLUSH_THRESHOLD to a value greater than the number of
 documents indexed (nor do I ever explicitly flush), the problem goes away.

 The query performed in step 4 depends on the contents of the document
 being indexed, so it varies.

 xapian-check indicates the database is consistent so it doesn't appear to
 be an on-disk corruption problem.

 After setting a breakpoint at backends/chert/chert_postlist.cc:153, here
 is a backtrace:

 {{{
 #0  report_read_error (position=0xa7b8175 "�\001\a�") at
 backends/chert/chert_postlist.cc:153
 #1  0xb7691677 in ChertPostList::move_to_chunk_containing (this=0xa0f37d8,
 desired_did=<value optimized out>)
     at backends/chert/chert_postlist.cc:876
 #2  0xb76916ff in ChertPostList::jump_to (this=0xa0f37d8, desired_did=475)
 at backends/chert/chert_postlist.cc:971
 #3  0xb76918ea in ChertPostListTable::get_doclength (this=0x8c2ca98,
 did=475, db=@0xbf901fa8)
     at backends/chert/chert_postlist.cc:68
 #4  0xb7678515 in ChertDatabase::get_doclength (this=0x8c2ca70, did=475)
 at backends/chert/chert_database.cc:797
 #5  0xb7678f4f in ChertWritableDatabase::get_doclength (this=0x8c2ca70,
 did=475)
     at backends/chert/chert_database.cc:1501
 #6  0xb768ed2b in ChertPostList::get_doclength (this=0xa7166c0) at
 backends/chert/chert_postlist.cc:726
 #7  0xb768cdb8 in ChertModifiedPostList::get_doclength (this=0xa7166c0)
     at backends/chert/chert_modifiedpostlist.cc:68
 #8  0xb75ecb6e in LeafPostList::get_weight (this=0xa7166c0) at
 api/leafpostlist.cc:77
 #9  0xb772fdfb in MultiAndPostList::get_weight (this=0xa7b8188) at
 matcher/multiandpostlist.cc:165
 #10 0xb772af34 in SelectPostList::get_weight (this=0xa7b8270) at
 matcher/selectpostlist.h:52
 #11 0xb7732da8 in MultiMatch::get_mset (this=0xbf9025dc, first=0,
 maxitems=5, check_at_least=5, mset=@0xbf902730,
     stats=@0xbf902644, mdecider=0x0, matchspy_legacy=0x0, sorter=0x0) at
 matcher/multimatch.cc:574
 #12 0xb7600bba in Xapian::Enquire::Internal::get_mset (this=0xa7004c0,
 first=0, maxitems=5, check_at_least=0,
     rset=0x0, mdecider=0x0, matchspy_legacy=0x0) at api/omenquire.cc:670
 #13 0xb76010f9 in Xapian::Enquire::get_mset (this=0xa3fafe8, first=0,
 maxitems=5, check_at_least=0, rset=0x0,
     mdecider=0x0) at api/omenquire.cc:1034
 #14 0xb7818257 in _wrap_Enquire_get_mset__SWIG_4 (nobjs=3,
 swig_obj=0xbf902798) at modern/xapian_wrap.cc:22653
 #15 0xb7819719 in _wrap_Enquire_get_mset (self=0x0, args=0x8e4cc34) at
 modern/xapian_wrap.cc:22798
 #16 0x0805cb97 in PyObject_Call ()
 [...]
 }}}
 (All frames over 16 are Python as I am using the Python bindings, so I've
 omitted them for brevity.)

-- 
Ticket URL: <http://trac.xapian.org/ticket/455>
Xapian <http://xapian.org/>
Xapian