[Xapian-discuss] weak populated b-trees?

[Olly Betts]

On Fri, Sep 05, 2008 at 08:14:21PM +0200, Markus Wörle wrote:
> So my question is: Is this reasonable? Respectively, do you have any  
> idea why my b-trees are such empty? Does Xapian merge weakly populated  
> blocks again?

We don't currently ever merge under-populated blocks, unless they become
totally empty.  We also don't ever shrink the file, even if blocks at
the end become free.

Martin Porter found that merging blocks tended to take a lot of effort
and usually they'd get refilled soon enough.  That was many years ago
though, and I've wondered before if we should revisit this decision.

> I am currently planning to stop indexing once a day to run xapian- 
> compact, but I am uncertain if this whould "denaturate" the system. I  
> have many modifications, and althought "best indexing performance" is  
> not really a point in my use-case, I feel somehow bad about  
> manipulating a natural-balanced b-tree in a non-changing environment.  

For now I'd just compact the database after a lot of deletion if you
don't plan to add back in a similar amount of data.  Even if you're
planning to update further, the penalty for the extra block splitting
that will be needed for a while afterwards is likely to be minor.

Cheers,
    Olly