[Xapian-devel] How to build index faster?

[Olly Betts]

On Tue, Apr 01, 2014 at 06:55:50PM +0800, 没有锂称 wrote:
> when the database is very large,   the functiion "commit()"  and
> "add_document()" Spend a very long time.

So commit() does the work of actually writing out batched up postlist
changes to disk, so it's not really surprising that it take time.

And add_document() will automatically commit changes every
$XAPIAN_FLUSH_THRESHOLD documents added/replaced/deleted, and when
that happens, it will take the time an explicit call to commit()
would take.

> how do i make it faster?

You can set environmental variable XAPIAN_FLUSH_THRESHOLD to a
larger value if you have plenty of memory (the default is 10000).
Each commit() will take longer, but there will be fewer of them and
overall it should be more efficient.

You can also use the brass backend instead of chert, which should
be a bit faster here.

Cheers,
    Olly