[Xapian-discuss] Robust "online" xapian backup?

Jesper Krogh jesper at krogh.cc
Thu Oct 14 05:51:28 BST 2010


Hi.

I'm struggling a bit with getting the Xapian indexes safely stored on tape.
LVM snapshots is not really an option given the performance penalties
of that technology. And the index often encounter writes in the
time it takes for the backup to copy off the files. Thus the
index is potentially unsafe on tape.

I dont know much about Xapian's internal versioning of data, but
I suspect it is something like PostgreSQL's MVCC system. Would
so my theory was: If no ->flush "has ended" while the backup was
running, then the index would be safe, since the last thing to do
is more or less to set the revision flag to the new version, which
all have to co-exists with a fully functional version?

Could we make a hook in Xapian so the backup could signal
to the indexer that it is allowed to ->flush the index, but
should hold of the "revision flag" until the backup signals that
it is done..  Wouldn't that ensure a fully working copy all the time?

Just shoot .. I may just not understand enough of Xapian's internals.

-- 
Jesper



More information about the Xapian-discuss mailing list