[Xapian-devel] Re: document_id globally incrementing

[Alexander Lind]

Revision on this;
I am mistaken below about the 'global' incrementing of document_id:s
across split databases - however they become incremented like that when
combined in a read only database opener. I assume this is something
Xapian does out of necessity, and that doesn't constitute a problem
since the db:s have their true document_id:s set correctly.

My question then instead is, how come that even though I call
writabledatabase_delete_document()
with the correct index pointer for x sub-index, and a certain
document_id within that index, that the entry for this document_id still
seems to persist in this index?

It was the assumption of that this function indeed worked like I thought
it would that brought me to the erroneous conclusion about the
document_id:s below. I assumed that since the documents seemed to stay
in the indexes after I called the delete function that the document_ids
must not be matching.

Very grateful for anyones input or ideas on this.

Alec

Alexander Lind wrote:
> Hi All
>
> I have made my xapian indexer automatically create new indexes once it
> reaches X documents in each, and for each document that I add to each
> sub-index, I record its document_id and its index_id (relating to what
> index the document ended up in).
>
> writabledatabase_add_document()  returns document_id:s beginning from 0
> for each new index when you add new documents, like you would expect.
>
> So far all good.
>
> Here is the problem: when I search through the indexes together (using
> database_add_database() on each sub-index), all the document_id:s are
> numbered globally, so it seems while writabledatabase_add_document()
> reset the document_id counter for each subindex, in the subindexes they
> were never reset.
>
> So instead of having 10 sub-indexes with 50 documents in each,
> document_id:s ranging from 0-50 in each, I seem to end up with 10
> sub-indexes with 50 documents in each, document_id:s ranging from 0-49
> in subindex 1, 50-99 in subindex 2, and so on. This would not be a
> problem if writabledatabase_add_document() returned these globally
> incrementing document_id:s, but it doesn't.
>
> Sorry for the lenghty email here, but is this a bug or a feature or am I
> doing something wrong?
>
> Thank you so much for your help.
> Alec
>
>