[Xapian-discuss] Feature request: Ligthen pressure on backup

[Jesper Krogh]

James Aylett wrote:
> On Mon, Mar 24, 2008 at 07:07:38AM +0100, Jesper Krogh wrote:
> 
>> Another solution could be to let Xapian query several databases and 
>> "merge" the result. Then I could make a new database each day and merge 
>> once a week (or another timepattern that would fit the purpose).
> 
> Jesper - Xapian can query multiple databases. You have to manage
> yourself which database you write into, but a database per day or
> similar would allow this. (You could perhaps merge fully shortly
> before you would do a level 0 backup anyway.)
> 
> If you're doing this kind of index-to-new-then-merge strategy (which
> some people use for the different challenge of live indexing with high
> search load), then the xapian-compact(1) command will probably be
> helpful to you.
> 
> Note that if you ascribe external meaning to Xapian document ids (for
> instance referencing them in a relational database), you may need to
> change things a little (such as by bringing external ids into Xapian
> and storing them in the document data, ie reversing the dependency)
> because of the way multiple database support is implemented.

This is excactly what I do. Can you elaborate a bit more on this?

I currently do put in the database-id into the document, but it doesn't 
serve any purpose. I do put in an unique term into the term-set of the 
documents that make replace-by-term work flawlessly. Can I preserve this 
during a merge (that I have a "unique-term" in the documents)?

> You may want to look at:
> <http://xapian.org/docs/admin_notes.html#backup-strategies>
> 
> and
> 
> <http://xapian.org/docs/admin_notes.html#merging-databases>
> for some other notes that may be of use here.

Thanks alot for the hints.

-- 
Jesper Krogh