[Xapian-devel] Something to think about

[James Aylett]

On Fri, Oct 12, 2007 at 05:27:00PM +0100, Olly Betts wrote:

> > Could we have a mechanism where the size of each opened database is
> > taken into account, perhaps doubled to provide padding, and if any one
> > overflows its padding a new exception is raised? If that were an
> > optional strategy, it would work in the majority of cases (could even
> > be the default).
> 
> Obviously we could, but it wouldn't help in the common case where the
> database is opened afresh by each search process.

If the db is opened each time, the document ids aren't guaranteed to
be consistent anyway, so does that case actually matter?

> One approach (rather a long term one) is to allow arbitrary docids -
> then searching over multiple databases can simple prefix the docids from
> each.  Probably the main challenge there is to maintain the ability to
> store them compactly.

I think if we want to do something like that, we'd want to separate
the idea of a user-level identifying docid, managed by the system, and
an underlying db-internal id. (Kind of like oids in postgresql.)
Although that has some nasty consequences, so maybe not. Hmm :-/

J

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james at tartarus.org                               uncertaintydivision.org