[Xapian-devel] Re: document_id globally incrementing

[Alexander Lind]

And another revision (phue!):

I solved it. All I had to do was to cast the doucment_id as an int.
Xapian tried to delete the document via a term since I unwittingly
passed the document_id in as a string. Doh :p

Alec

Alexander Lind wrote:
> Revision on this;
> I am mistaken below about the 'global' incrementing of document_id:s
> across split databases - however they become incremented like that when
> combined in a read only database opener. I assume this is something
> Xapian does out of necessity, and that doesn't constitute a problem
> since the db:s have their true document_id:s set correctly.
>
> My question then instead is, how come that even though I call
> writabledatabase_delete_document()
> with the correct index pointer for x sub-index, and a certain
> document_id within that index, that the entry for this document_id still
> seems to persist in this index?
>
> It was the assumption of that this function indeed worked like I thought
> it would that brought me to the erroneous conclusion about the
> document_id:s below. I assumed that since the documents seemed to stay
> in the indexes after I called the delete function that the document_ids
> must not be matching.
>
> Very grateful for anyones input or ideas on this.
>
> Alec
>
> Alexander Lind wrote:
>   
>> Hi All
>>
>> I have made my xapian indexer automatically create new indexes once it
>> reaches X documents in each, and for each document that I add to each
>> sub-index, I record its document_id and its index_id (relating to what
>> index the document ended up in).
>>
>> writabledatabase_add_document()  returns document_id:s beginning from 0
>> for each new index when you add new documents, like you would expect.
>>
>> So far all good.
>>
>> Here is the problem: when I search through the indexes together (using
>> database_add_database() on each sub-index), all the document_id:s are
>> numbered globally, so it seems while writabledatabase_add_document()
>> reset the document_id counter for each subindex, in the subindexes they
>> were never reset.
>>
>> So instead of having 10 sub-indexes with 50 documents in each,
>> document_id:s ranging from 0-50 in each, I seem to end up with 10
>> sub-indexes with 50 documents in each, document_id:s ranging from 0-49
>> in subindex 1, 50-99 in subindex 2, and so on. This would not be a
>> problem if writabledatabase_add_document() returned these globally
>> incrementing document_id:s, but it doesn't.
>>
>> Sorry for the lenghty email here, but is this a bug or a feature or am I
>> doing something wrong?
>>
>> Thank you so much for your help.
>> Alec
>>
>>   
>>     
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.tartarus.org/pipermail/xapian-devel/attachments/20061202/c573a3a8/attachment.htm