[Xapian-discuss] can a corrupt database be fixed?

[Ronan]

Hi.

I have a Xapian database which appears to be corrupt.  I know this is
a long shot, but I'm wondering if there's any way to repair it.

Why do I think it's corrupt?  Let me present some evidence.

If I run `delve' on the database, its output is always the following
two lines:

number of documents = 559147 
Error: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation

If I try to copy the database using copydatabase, its output always is
the following line:

copydatabase: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation

If I run the Linux `fuser' command on the database's files, it always
tells me that they're not in use.

The database is used by PHP software running within Apache.  In Apache's
PHP error log from last night, I find a number of messages like the
following which mention "DatabaseCorruptError":

[23-Jul-2008 23:26:39] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `feeling' in CONCEALED on line 720

(I have replaced the filename with the string "CONCEALED".)

There are a total of 14 messages mentioning "DatabaseCorruptError",
which I have put in the attached file.

In that same PHP error log, there is a constant stream of messages about
"DatabaseModifiedError: The revision being read has been discarded".
(Although these messages have always been appearing, the rate has been
very high since last night.)  Also, the PHP application isn't working
properly at present.

Taken together, I think the above gives sufficient reason to conclude
that the database is corrupt.

I can probably regenerate the database starting from a backup of 3 months
ago, but it would be much easier if there was some program I could run
to repair it.

Alternatively, if there is some programmatic approach which would salvage
most of the data, that would also be good.  I am reasonably familiar
with Xapian's PHP API, so, if given an algorithm, I should be able to
implement it.

In case it's relevant: the version of the Xapian software I'm using
is 0.9.9.  To be precise: I'm using a fairly old Debian system, and the
relevant Debian packages seem to be libxapian13, which is at version
0.9.9-0stable1, and php4-xapian, which is at version 0.9.9-1stable1.

Regards,

Ronan.
-------------- next part --------------
[23-Jul-2008 23:26:39] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `feeling' in CONCEALED on line 720
[23-Jul-2008 23:27:12] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `far' in CONCEALED on line 720
[23-Jul-2008 23:29:16] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:30:04] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `far' in CONCEALED on line 660
[23-Jul-2008 23:32:01] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `far' in CONCEALED on line 720
[23-Jul-2008 23:36:28] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `feeling' in CONCEALED on line 720
[23-Jul-2008 23:36:56] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `referendum' in CONCEALED on line 720
[23-Jul-2008 23:37:26] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:49:05] PHP Fatal error:  DatabaseCorruptError: Db block overwritten - are there multiple writers? in CONCEALED on line 222
[23-Jul-2008 23:50:28] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:51:51] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:53:43] PHP Fatal error:  DatabaseCorruptError: Unexpected end of posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:54:03] PHP Fatal error:  DatabaseCorruptError: Db block overwritten - are there multiple writers? in CONCEALED on line 222
[23-Jul-2008 23:54:05] PHP Fatal error:  DatabaseCorruptError: Db block overwritten - are there multiple writers? in CONCEALED on line 222