DatabaseModifiedError, how to handle it correctly when searching?

[Olly Betts]

On Mon, Aug 30, 2021 at 06:50:48PM -0700, Yuan Fu wrote:
>   int try_count = 0;
>   while (true)
>     {
>       try
>         {
>           (do stuff)
>           break;
>         }
>       catch (Xapian::DatabaseModifiedError &e)
>         {
>           if (try_count > 0) throw e;
>           database.reopen();
>           try_count++;
>         }
>     }

You should never get DatabaseModifiedError for a database you have a
write lock on - i.e. a WritableDatabase (because what
DatabaseModifiedError means is that a writer has committed modifications
to the database underneath you in such a way that the operation you
asked for can't be performed, and Xapian doesn't support concurrent
writers so if you are the current writer this can't happen).

Also, note that if the writer is making a series of small commits in
quick succession you could need to retry more than once.

It also often makes sense to retry a group of operations - e.g. if you
perform a search and hit DatabaseModifiedError reading the documents
in the results, you probably want to retry the search starting from
get_mset() (even though that part worked) so that your results are from
a consistent database state (this is the main reason that there isn't an
auto-retry in this situation).  DatabaseModifiedError should be a rare
occurrence so redoing extra in response to it shouldn't be costly.

There is a longer term plan to allow databases to keep old revisions
which readers are using around, which will eventually provide a way to
prevent this exception.

Cheers,
    Olly