errors on rebuild

[Ryan Cross]

Hello,

I am trying to rebuild an index of 2+ million documents and have not been successful.  I am running 

Python 2.7
Django 1.7
Haystack 2.1.1
Xapian 1.2.21

The index rebuild command I’m using is: django-admin.py rebuild_index --noinput --batch-size=100000
The rebuild completes but an immediate xapian-check returns this error:

xapian-check ./archive_index
record:
baseB blocksize=8K items=2219301 lastblock=126957 revision=223 levels=2 root=126448
B-tree checked okay
record table structure checked OK

termlist:
baseB blocksize=8K items=4438602 lastblock=1134080 revision=223 levels=3 root=1128535
B-tree checked okay
termlist table structure checked OK

postlist:
baseB blocksize=8K items=34620971 lastblock=2286112 revision=223 levels=3 root=3577
B-tree checked okay
postlist table structure checked OK

position:
baseB blocksize=8K items=942481128 lastblock=2787202 revision=223 levels=3 root=2776373
xapian-check: DatabaseCorruptError: Expected block 2763648 to be level 0, not 1


Trying the latest stable version, Xapian 1.4.3, it fails during the rebuild:

All documents removed.
Indexing 2233651 messages
Traceback (most recent call last):
…

  File "/a/mailarch/current/haystack/management/commands/update_index.py", line 221, in handle_label
    self.update_backend(label, using)
  File "/a/mailarch/current/haystack/management/commands/update_index.py", line 266, in update_backend
    do_update(backend, index, qs, start, end, total, self.verbosity)
  File "/a/mailarch/current/haystack/management/commands/update_index.py", line 89, in do_update
    backend.update(index, current_qs)
  File "/a/mailarch/current/haystack/backends/xapian_backend.py", line 286, in update
    database.close()
xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1
docdata:
blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410

Any suggestions for how I could get more information to troubleshoot this failure would be 
greatly appreciated.

Thank You,
Ryan