[Xapian-devel] Problem in Indexing

[Parth Gupta]

Hello All,

I am trying to index a collection of files. Details of the collection is
given below.

Types of Files: text files with .txt extension
Size of the collection: 11400 documents [1.6 GB]

This takes a lot of time to index and indexing for last 20 hrs or so. I am
using omindex.

I notice that around 2900 docs are indexed very smoothly and suddenly after
that indexing becomes very sluggish.

I have tried couple of tricks like replacing the index_text() call to
index_text_without_positions(). I also tried after setting the
XAPIAN_FLUSH_THRESHLOD to 1500 documents from 10000 default. Above mentioned
time is after this tricks.

Any help will be appreciated.

Thanks,
Parth.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20110504/12d05457/attachment.htm>