[Xapian-tickets] [Xapian] #317: Database corruption after disk-full error

[Xapian]

#317: Database corruption after disk-full error
---------------------------+------------------------------------------------
 Reporter:  richard        |        Owner:  richard
     Type:  defect         |       Status:  new    
 Priority:  normal         |    Milestone:  1.0.10 
Component:  Backend-Flint  |      Version:  1.0.7  
 Severity:  normal         |   Resolution:         
 Keywords:                 |    Blockedby:         
 Platform:  All            |     Blocking:         
---------------------------+------------------------------------------------

Comment(by richard):

 OK, so, if I run, with SVN trunk, the attached xaptest.py script in an
 ext3 partition with slightly over 100Mb of free space available (102754kb,
 to be precise), I get 6000 documents successfully indexed, and then get
 errors due to running out of space, as follows (end of log shown - before
 this it just counts up to 6600):

 {{{
 6610
 6620
 6630
 6640
 6650
 6660
 6670
 6680
 6690
 Error writing to file (No space left on device)
 Error writing to file (No space left on device)
 Traceback (most recent call last):
   File "/home/richard/xaptest.py", line 17, in <module>
     db.add_document(doc)
 xapian.DatabaseError: Error writing to file (No space left on device)
 }}}

 The process terminates at this point, and the database is left in a
 consistent state.

 However, if I then re-run the xaptest.py script with the same database, I
 get:

 {{{
 0
 Modifications failed (DatabaseError: Error writing to file (No space left
 on device)), and cannot set consistent table revision numbers: Couldn't
 reread base A
 10
 20
 30
 40
 [...]
 650
 660
 670
 680
 690
 Couldn't reread base A
 Traceback (most recent call last):
   File "/home/richard/xaptest.py", line 17, in <module>
     db.add_document(doc)
 xapian.DatabaseCorruptError: Couldn't reread base A
 Segmentation fault
 }}}
 (Where [...] was the counts from 50 to 640)

 The database is left in a corrupt state: here's a listing of the database
 directory:
 {{{
 total 102778
 -rw-r--r-- 1 richard richard        0 2008-12-18 01:23 flintlock
 -rw-r--r-- 1 richard richard       12 2008-12-18 01:22 iamflint
 -rw-r--r-- 1 richard richard      951 2008-12-18 01:22 position.baseB
 -rw-r--r-- 1 richard richard 68071424 2008-12-18 01:23 position.DB
 -rw-r--r-- 1 richard richard 35815424 2008-12-18 01:23 postlist.DB
 -rw-r--r-- 1 richard richard        0 2008-12-18 01:23 postlist.tmp
 -rw-r--r-- 1 richard richard       16 2008-12-18 01:22 record.baseB
 -rw-r--r-- 1 richard richard    98304 2008-12-18 01:23 record.DB
 -rw-r--r-- 1 richard richard       25 2008-12-18 01:22 termlist.baseB
 -rw-r--r-- 1 richard richard   745472 2008-12-18 01:23 termlist.DB
 -rw-r--r-- 1 richard richard       16 2008-12-18 01:22 uuid
 -rw-r--r-- 1 richard richard       16 2008-12-18 01:22 value.baseB
 -rw-r--r-- 1 richard richard    98304 2008-12-18 01:23 value.DB
 }}}

 ... we can see that there is no base file for the postlist table.

 This corruption doesn't happen if I change the xaptest.py script so that
 it aborts after an error.

-- 
Ticket URL: <http://trac.xapian.org/ticket/317#comment:1>
Xapian <http://xapian.org/>
Xapian