errors on rebuild

[Olly Betts]

On Sun, Apr 02, 2017 at 10:40:22PM -0500, Bron Gondwana wrote:
> The trick that FastMail/Cyrus IMAPd uses of batching to smaller indexes
> and then compacting a few of them together at once may be interesting as
> well.  I have no idea how it performs on really massive indexes though,
> because we index per user.

Yes, that's a good approach for a large take-up, though may be harder
to implement with a middle layer (xapian-haystack in this case) than
if you're using xapian directly.

It should scale well - the old gmane search used this, and indexed
well over 100 million documents.

Cheers,
    Olly