[Xapian-discuss] Re: Boolean merging of DB's

[Olly Betts]

On Fri, Aug 03, 2007 at 06:42:03AM +0000, David Morris wrote:
> David Morris <dmorris <at> sirca.org.au> writes:
> > I'm wondering if there is a xapian-compact tool that lets me do boolean merging
> > of  2 xapian db's into one db.
> > 
> > Take this scenario: I have db 'A' with 100 doc's and db 'B' with 30 doc's. Most
> > of the docs in B will also be in A, but with newer content, so when I merge
> > them, I want to make a db 'C' with everything in A, but anything newer that's
> > contained in B will overwrite any A entries.
>
> Actually, please ignore me, this doesn't quite make sense. There's no way to
> know the mapping of a document in one db to the "same" document in another db...

Well, you could store have a unique term per document and use those.

But xapian-compact can't really do this sort of thing.  It's fast
because documents from each database are just copied over with document
ids adjusted by the addition or subtraction of a constant for each
database, so it doesn't need to unpack and repack posting lists, it can
just adjust the Btree keys and the start of the tags.

> It's all good, I can make a custom merge app that can do this...

That's what you'll need to do.  Perhaps examples/copydatabase.cc in
xapian-core is a useful starting point.

Cheers,
    Olly