[Xapian-discuss] Repairing corrupt search DB

[Richard Boulton]

2009/8/25 Alex Viggio <aviggio at bivio.biz>

> Olly Betts wrote:
> > This recovery should have happened automatically, and it would be useful
> > to understand why it didn't in this case.
> >
> > Do you have the original base files still?  If so, how large are they?
>
> Yes I backed that up. Unfortunately the Xapian data tarball is 7.3G
> (4.1G gzip'd). This includes all of the .base[AB] and .DB files in the
> corrupted state:
>
>   929 Aug 24 12:13 value.baseA
>  2.3G Aug 24 12:13 postlist.DB
>   37K Aug 24 12:13 postlist.baseA
>   16M Aug 24 12:13 record.DB
>    12 Aug 24 12:13 iamflint
>   929 Aug 24 12:14 value.baseB
>  989M Aug 24 12:14 termlist.DB
>   16K Aug 24 12:14 termlist.baseB
>   16K Aug 24 12:14 termlist.baseA
>   269 Aug 24 12:14 record.baseB
>   64K Aug 24 12:14 position.baseB
>   64K Aug 24 12:14 position.baseA
>     0 Aug 24 12:14 flintlock
>   57M Aug 24 12:14 value.DB
>   269 Aug 24 12:14 record.baseA
>  4.0G Aug 24 12:18 position.DB
>
> > 1.0.10 fixed an issue which could lead to DatabaseCorruptError if the
> > disk filled up.  There hasn't been anything since then.
>
> I checked the 1.0.13 and .14 release overviews, and that appeared to be
> the case.
>

If you could make a tarball for us containing just the .baseA and .baseB
files, this might help us to work out what the problem was (ie, why the
automatic recovery didn't work).  The .base files are tiny compared to the
.DB files - ideally, attach the files to a ticket in xapian-trac, but if you
prefer, send it to Olly or me by private email (the tarball might be too big
to be permitted on the list, still).

The .base[AB] files contain details of which blocks are used in the
corresponding .DB file, which the root block is, and which revision the
latest revision in the .DB file is, but no actual data about which
terms/values/etc are in the database, so there shouldn't be any confidential
information in there.

-- 
Richard