[Xapian-discuss] How to speed up indexing ?

[Charlie Hull]

cel tix44 wrote:
> I'm new to Xapian & need some help, many thanks if anyone replies.
> 
> I did a release build from xapian-core-1.0.7 with VS2008 by using
> Charlie Hull's makefiles.
> 
> I'm trying to test-index my dataset -- some 200'000 docs, each
> document being (on average) 50 bytes long and having 6 words.
> 
> I tried (a) not to use stemmer, (b) commit_transaction() on every
> 50/100/etc. docs, (c) not to use transactions at all -- but in all
> scenarios indexing goes at ~10 doc/sec or 500 bytes per second.
> 
> This should probably be ~400 times faster, I'm clearly doing something
> wrong. Can anyone give me a hint or direct me to a source on the net
> to do some reading?

If you could let us know the platform you're using, and how you're 
accessing Xapian (which bindings for example, or directly using C/C++?), 
and even post the code you're using for your indexer, that would help 
hugely.

Cheers

Charlie

> 
> Regards
> Celto
> 
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>