Using multiple temporary indexes during updates
Jean-Francois Dockes
jf at dockes.org
Fri Mar 15 19:15:55 GMT 2024
Hi,
I have been playing at converting the index update stage of the Recoll indexer to use
multiple temporary indexes and a final merge.
This yields an improvement factor of almost 3 (on my quad-core CPU), for the total
indexing time for "easy" files like HTML pages. This is nice (!) and I wanted to share my
admiration for the "compact()" method.
If someone is interested in a bit more detail:
https://www.recoll.org/pages/idxthreads/threadingRecoll.html#_the_xapian_bottleneck_and_how_it_was_resolved_thanks_to_xapian
Cheers,
jf
More information about the Xapian-discuss
mailing list