[Xapian-discuss] How to speed up indexing ?

cel tix44 celtix44 at gmail.com
Thu Aug 21 10:17:00 BST 2008


I'm new to Xapian & need some help, many thanks if anyone replies.

I did a release build from xapian-core-1.0.7 with VS2008 by using
Charlie Hull's makefiles.

I'm trying to test-index my dataset -- some 200'000 docs, each
document being (on average) 50 bytes long and having 6 words.

I tried (a) not to use stemmer, (b) commit_transaction() on every
50/100/etc. docs, (c) not to use transactions at all -- but in all
scenarios indexing goes at ~10 doc/sec or 500 bytes per second.

This should probably be ~400 times faster, I'm clearly doing something
wrong. Can anyone give me a hint or direct me to a source on the net
to do some reading?

Regards
Celto



More information about the Xapian-discuss mailing list