[Xapian-discuss] DatabaseModifiedError

Chris chris at s-4-u.net
Wed Mar 23 10:57:10 GMT 2011


On 03/23/2011 11:35 AM, Richard Boulton wrote:
> On 23 March 2011 10:16, Andrew Betts <andrew.betts at assanka.net> wrote:
>> Hi,
>>
>> I am starting to see more frequent occurrences of a DatabaseModifiedError when users execute searches on my site.  Since our index is being updated all the time, its easy to understand why these happen, but is it possible to ignore this error and execute the search against the stale revision?  It's likely only a few seconds out of date, and if I follow the advice and reopen() the database, it's quite possible that it will have been modified again before I manage to call get_mset.
> Xapian can't be told to ignore the error and use the stale revision,
> because the error is only thrown when the data Xapian needs from the
> old revision has already been overwritten by new data.  If you're
> searching on version N, you won't get the error until before version
> N+1 has been committed.  After version N+1 has been committed, changes
> to the database (ie, leading to version N+2) will start overwriting
> blocks which were used in version N (but not in version N+1), and
> there is then a risk of the exception being thrown.
>
> Assuming you're updating the index every few seconds, and searches
> take no more than a couple of seconds in the worst case you could
> perhaps insert a pause after committing a change before performing any
> further modifications to the database, to allow readers a chance to
> reopen on the new revision before risking overwriting blocks from the
> old revision.
>
We had the same problem and i hacked around it by a) using Xapian
transactions to get a bigger window and b) catch the exception, reopen
reader database and rerun the search upto 5 times, which works well so
far, even if i suspect that this error might hit us again when the
number of searches increase (in which case i'll just raise the number of
repeats).
it's not an optimal solution, but a workable.

greets, Chris



More information about the Xapian-discuss mailing list