Using multiple temporary indexes during updates

Jean-Francois Dockes jf at dockes.org
Fri Mar 15 19:15:55 GMT 2024


Hi,

I have been playing at converting the index update stage of the Recoll indexer to use
multiple temporary indexes and a final merge.

This yields an improvement factor of almost 3 (on my quad-core CPU), for the total
indexing time for "easy" files like HTML pages. This is nice (!) and I wanted to share my
admiration for the "compact()" method.

If someone is interested in a bit more detail:
https://www.recoll.org/pages/idxthreads/threadingRecoll.html#_the_xapian_bottleneck_and_how_it_was_resolved_thanks_to_xapian

Cheers,

jf




More information about the Xapian-discuss mailing list