[Xapian-discuss] last_mod performance

[Olly Betts]

On Fri, Feb 20, 2009 at 04:57:37PM +1030, Frank J Bruzzaniti wrote:
> I found that last mod from that patch ends up deleting documents.
> But I found another patch on trac that worked, I've been testing it and 
> have found indexing time for me went down from 4 hours to about 8 minutes.

Clearly checking the last modified time is going to be a win when
nothing has changed.

The more interesting questions are what the overhead is when all the
documents need reindexing anyway, and if that overhead is measurable
what the break-even point is in terms of the proportion of documents
which need to have changed to make it a win.

If the overhead isn't measurable, this can just be always on.  If
it's significant, then perhaps the option should default off.

> I use a slightly modified version in here.
> 
> http://trac.xapian.org/attachment/ticket/290/office2007.patch

It would be helpful to have the patch you're actually testing, without
any other unrelated changes

Cheers,
    Olly