[Xapian-discuss] "DatabaseCorruptError: Cannot open tables at consistent revisions"

[Olly Betts]

On Wed, Apr 29, 2009 at 12:37:03PM +0930, Frank J Bruzzaniti wrote:
> I use omindex then I do a merge from 2 databases to "default" after each 
> crawl.
> 
> I gather it's the merge that's doing it.

Yes, that'll be it.  I guess you're currently merging directly to
"default" or using a symlink for default?  The former doesn't work
because the merged database isn't valid until the merge is complete.
The latter is problematic because Xapian can open some tables from the
old path and some from the new, and so get an inconsistent set.

The way to get this to work without such problems is to use a stub
database file for default which points to the current actual database -
this is just a text file containing:

auto /path/to/merged.0

Then to update, you would do:

rm -rf /path/to/merged.1
xapian-compact /path/to/merged.0 /path/to/incremental /path/to/merged.1
echo 'auto /path/to/merged.1' > default.new
# Make sure that "default.new" is on the same partition as "default".
mv default.new default

For the next update, swap 1 and 0 in the above, so the true database is
alternately named "/path/to/merged.0" and "/path/to/merged.1".

Sadly this approach doesn't quite work on Microsoft Windows which doesn't
support POSIX semantics for atomic update of files by renaming like
this.

Cheers,
    Olly