[Xapian-discuss] Repairing corrupt search DB

[Olly Betts]

On Mon, Aug 24, 2009 at 02:44:43PM -0600, Alex Viggio wrote:
> It seemed worth trying some quick fixes given a full Xapian db rebuild
> on this host and production data runs several days. Got somes ideas from 
> this 2008 post by Olly:
> 
> http://article.gmane.org/gmane.comp.search.xapian.general/6333

Yes, that's still a useful overview.

> Results of my xapian-check run copied below. Since it reported mostly 
> success with the "baseB" files, I first tried moving aside the Xapian 
> *.baseA db files with the exception of postlist.baseA. FWIW, there was 
> no postlist.baseB file in existence. A possible clue? Let me know if 
> anything else stands out.

I should point out that currently xapian-check checks the latest version
of each table individually, even though opening the database would want
a consistent revision for each table.

So the reason you see baseB for most tables is that baseB is newer.  The
lack of a baseA for postlist isn't paticularly significant - it probably
just means that the postlist table was modified so its baseA revision
is no longer valid.

> I then tried some Xapian read/write ops and still got the "Cannot open 
> tables at consistent revisions" error in all test cases.

Since there's no postlist.baseB, you aren't going to be able to open the
baseB revision of the database.

> Next I tried moving back the *.baseA files and moving aside all of the
> *.baseB files. Success -- or so it appears! Both the search rebuild 
> (writes) and web app search (reads) worked.

This recovery should have happened automatically, and it would be useful
to understand why it didn't in this case.

Do you have the original base files still?  If so, how large are they?

> I have seen a couple of Xapian db corruptions in the past, but this was 
> the first instance since we upgraded to Xapian C++ core v1.0.12 and 
> Search::Xapian Perl XS v1.0.12.0.

1.0.10 fixed an issue which could lead to DatabaseCorruptError if the
disk filled up.  There hasn't been anything since then.

Cheers,
    Olly