[Xapian-discuss] Exception: Key too long

[Markus Wörle]

Hi!

I trying to add a large amount of documents to a database. After about 
33400 documents I get the following exception for each new document:

Key too long: length was 254 bytes, maximum length of a key is 
BTREE_MAX_KEY_LEN

I am pretty sure that this is not my fault. In my code, I limited the 
maximum term-length to 200 bytes (and no \0 bytes), and one of the 
documents where the error occured has a maximum-term-lengh of 11. If 
I'm trying to add one of the documents that thows these exceptions to 
an empty database, everything works fine.

This is weird because I tested xapian long before with a much bigger 
abount of dummy-data (about 150.000 documents) with no problems. Now, 
during the final test with real-world-data this exception happens with 
no (to me) explainable reason.

I am using libxapian11 version: 0.9.6-4.99dapper, and the 0.9.6 
perl-bindings (but I dont think the problem is with these bingings).

quarzcheck on the odd database says:

record:
baseA blocksize=8K items=33425 lastblock=10010 revision=2728 levels=2 
root=8991
B-tree checked okay
record table structure checked OK

termlist:
baseA blocksize=8K items=33424 lastblock=4873 revision=2728 levels=2 
root=2252
B-tree checked okay
termlist table structure checked OK

postlist:
baseA blocksize=8K items=337722 lastblock=6406 revision=2728 levels=2 
root=16
B-tree checked okay
postlist table structure checked OK

position:
baseA blocksize=8K items=2381373 lastblock=6595 revision=2728 levels=2 
root=2311
B-tree checked okay
position table structure checked OK

value:
baseA blocksize=8K items=0 lastblock=0 revision=2728 levels=0 
root=(faked)
void B-tree checked okay
value table structure checked OK

No errors found

I'm trying to isolate this incident further, but at the moment I am 
pretty clueless and even don't have an idea where to start.

Do you have any ideas?

Regards,
mrks