[Xapian-discuss] Optimal usage of xapian-compact for merging

Henry C. henka at cityweb.co.za
Tue Feb 2 12:49:46 GMT 2010


Greets,

I've been wondering, what's the sane/optimal use of xapian-compact when
merging many indexes with a view to maximum merging performance?

The obvious:
- only use -F on the final db.
- use -m since I'm merging more than 3 dbs.

Best strategy?
a)  loop:  merge batches (of say 50, where the individual db's are small)
into a temp index, then merge the (larger) temp into the final product...
end-loop

b)  loop:  merge batches (of say 50, where the individual db's are small)
into many temp indexes... end-loop
Then merge those (larger) temps into the final product.

Finally, presumably it's best to use the same blocksize (-b) as the
underlying filesystem?  I see the default is 8K, but the default blocksize
on (eg) ext3 is 4k...  or am I way off here?

Thanks
Henry




More information about the Xapian-discuss mailing list