[Xapian-devel] Full MVCC in Brass

[Olly Betts]

On Wed, Aug 20, 2014 at 10:46:24PM -0400, Austin Clements wrote:
> On Wed, 20 Aug 2014, Adam Sjøgren <asjo at koldfront.dk> wrote:
> > Austin Clements <aclements at csail.mit.edu> writes:
> >
> >> Full MVCC would enable Xapian to keep any database revision valid as
> >> long as any reader is using it, thus eliminating the dreaded
> >> DatabaseModifiedError and simplifying application logic.
> >
> > This would also make it easier to spool a consistent backup to tape of
> > an index that is being used (updated), right?
> 
> As Olly mentioned, this wouldn't ensure that a copy straight from the
> filesystem would be a valid database.  However, it would be pretty
> close.  The problem is that, even if you're holding a read lock on a
> given revision, the database metadata in "iambrass" may no longer be
> consistent with that revision.  However, I think if you were to save the
> iambrass file as it was when the reader opened and locked its revision,
> you could then safely copy the remaining files directly from the file
> system.

Ah yes, that's nice and simple, and I think should work.

We could provide a helper tool which did something like opening the
database with a read lock, taking a exclusive lock on "iambrass.saved" and
saving the version file data for the opened revision to it.  This tool
could run as a wrapper around the backup command.  Then after a restore,
you should just need to do this to get a working database:

mv iambrass.saved iambrass

Even if there's some subtlety I'm missing, we should certainly try to
provide a way to make backups more easily.

Cheers,
    Olly