[Xapian-devel] Re: document_id globally incrementing

[Olly Betts]

On Sat, Dec 02, 2006 at 10:16:40PM -0800, Alexander Lind wrote:
> writabledatabase_add_document()  returns document_id:s beginning from 0
> for each new index when you add new documents, like you would expect.

Docid 0 is invalid - the returned docids should start at 1.  Perhaps
that's what you meant, but I thought I should clarify.

> So instead of having 10 sub-indexes with 50 documents in each,
> document_id:s ranging from 0-50 in each, I seem to end up with 10
> sub-indexes with 50 documents in each, document_id:s ranging from 0-49
> in subindex 1, 50-99 in subindex 2, and so on. This would not be a
> problem if writabledatabase_add_document() returned these globally
> incrementing document_id:s, but it doesn't.

When you search over a combination of databases, the document ids are
simply interleaved to avoid clashes.  You can easily reverse the
formula used:

http://article.gmane.org/gmane.comp.search.xapian.general/1375

When writing, you're just accessing the docids in a single database.  It
would be feasible to allow writing to multiple database managed by a
single object, but nobody's yet done the work required to support this.
There was some discussion of that in this thread:

http://thread.gmane.org/gmane.comp.search.xapian.general/3464

Cheers,
    Olly