[Xapian-discuss] My new record: Indexing 20 millions docs = 79m9.378s

[Kevin Duraj]

Gentoo Linux 2.6
8 AMD Opteron 64-bit Processors
32GB Memory
--------------------------------------------------------------------------------

Environment:
------------------
XAPIAN_FLUSH_THRESHOLD=21000000
XAPIAN_FLUSH_THRESHOLD_LENGTH=16000000
XAPIAN_PREFER_FLINT=True
Indexing 20 million documents:
--stemmer=none
-------------------------------------------
real    79m9.378s
user    77m28.696s
sys     1m36.654s

# delve /home/kevin/index
---------------------------------------
number of documents = 19999995
average document length = 8.18631


PS: In my scenario after 25 million records the indexing significantly slows
down (2x-4x)
I do not know why? Could it be because of the B-Tree become very complex?

- Kevin Duraj