[Xapian-tickets] [Xapian] #317: Database corruption after disk-full error
Xapian
nobody at xapian.org
Thu Dec 18 03:35:50 GMT 2008
#317: Database corruption after disk-full error
---------------------------+------------------------------------------------
Reporter: richard | Owner: richard
Type: defect | Status: new
Priority: normal | Milestone: 1.0.10
Component: Backend-Flint | Version: 1.0.7
Severity: normal | Resolution:
Keywords: | Blockedby:
Platform: All | Blocking:
---------------------------+------------------------------------------------
Old description:
> I've recently been testing the behaviour of xapian when the disk becomes
> full, after reports of corruption at a customer site in this situation,
> by performing some indexing to a database in a small partition.
>
> The key seems to be that, if a WritableDatabase is re-used after an
> operation with it has encountered an IOError, all sorts of corruption is
> possible. I've got a python script which repeatably produces a corrupt
> database when run in a small partiton, which I'll attach here shortly.
> However, the exact mode of failure is very sensitive to the initial
> amount of space available.
>
> I've only tested this with the flint backend so far, and only with xapian
> 1.0.7 (the version in ubuntu hardy) but it's likely that chert and more
> recent xapian's have a similar problem.
New description:
I've recently been testing the behaviour of xapian when the disk becomes
full, after reports of corruption at a customer site in this situation, by
performing some indexing to a database in a small partition.
The key seems to be that, if a !WritableDatabase is re-used after an
operation with it has encountered an IOError, all sorts of corruption is
possible. I've got a python script which repeatably produces a corrupt
database when run in a small partiton, which I'll attach here shortly.
However, the exact mode of failure is very sensitive to the initial amount
of space available.
I've only tested this with the flint backend so far, and only with xapian
1.0.7 (the version in ubuntu hardy) but it's likely that chert and more
recent xapian's have a similar problem.
--
Comment(by olly):
Hmm, what is actually the first system call which fails due to the full
disk in this scenario?
Judging from the patch, it seems it is cancel() which must be throwing,
but that only seems to read from disk so I'm not sure why it would fail in
this situation...
The fix does indeed look promising, though I'm not totally sure it's
exactly the right (or at least best) approach (perhaps we should be more
fine-grained about where the exception happened), and also it seems that
if we throw in the new code, we're probably no better off...
--
Ticket URL: <http://trac.xapian.org/ticket/317#comment:3>
Xapian <http://xapian.org/>
Xapian
More information about the Xapian-tickets
mailing list