[Xapian-discuss] BTREE_MAX_KEY_LEN=252

Olly Betts olly at survex.com
Tue May 1 03:27:31 BST 2007


On Tue, May 01, 2007 at 11:16:35AM +0900, Josef Novak wrote:
> Exception: Key too long: length was 446 bytes, maximum length of a key
> is BTREE_MAX_KEY_LEN bytes
> 
>  If I make sure to check that the new posting token (key) does not
> exceed the 252 byte maximum specified here,
> http://www.xapian.org/docs/sourcedoc/html/btree_8h.html#b8d8c0c3cbbcec113aa5e3f5edace5dd

The btree keys need to contain a term and other information, so the
actual safe term length limit is less than 252 bytes - I recommend
imposing a limit of 240.  Read this for the full details:

http://article.gmane.org/gmane.comp.search.xapian.general/3656

> I have no problems.  However, I noticed that if I run the program
> without checking the posting token length before attempting to add it,
> it will sometimes throw the exception and keep on trucking, yet
> sometimes it will throw the exception and then throw a segmentation
> fault and unceremoniously die.

You shouldn't get a SEGV, so that sounds like a bug.  Can you provide a
small self-contained example which demonstrates this?

>  As far as I can tell it is the over-long posting token that is
> causing the problem in both cases.  It may be that the exception then
> causes the posting index to also get out of sync?

The exception should cause the current batch of unapplied changes to be
abandonned, so it shouldn't be possible for the tables to get out of
sync.

Cheers,
    Olly



More information about the Xapian-discuss mailing list