[Xapian-discuss] Feature request: Ligthen pressure on backup

[Richard Boulton]

Kevin Duraj wrote:
> I hate to distract community with and old issue about dropping Quartz
> database in Xapian 1.0 and adopting Flint database. Several of us
> posted threads that when we upgraded to Xapian 1.0 converting to Flint
> database we noticed severe degradation in indexing performance.

I'm pretty sure it was only you who reported degradation of the 
performance.  We asked for more details, and for you to help us to 
reproduce the behaviour so that we could investigate it, but these 
haven't been forthcoming.  All our tests have indicated better 
performance with Flint - so there must be something specific about the 
way in which you use Flint that is causing the problem.  The correct way 
forward is to identify what's special about your use-case, and to fix 
whatever problem in Flint it is that is causing you problems.

> We were told that Flint database is more compact than Quartz database
> and uses less disk space and we must sacrificed some performance
> because of it.

No - flint is indeed more compact, and this should _improve_ 
performance, due to reduced IO.  I've not seen any situation where flint 
is slower than quartz was; if we could find and reproduce such a 
situation, we'd be able to investigate and fix it.

 > It seems that this might not be entirely true, because
> after some inserts and deletes from Flint database, Flint database
> grows exponentially to the size of Quartz database and often must be
> compact. Looks like we do not have smaller database with Flint
> database than we had with Quartz database, but for sure we lost
> significant performance during indexing.

As requested several times in the past, please, please, either give us 
the information we need to reproduce this ourselves, or stop making this 
claim.  Personally, my tests showed large performance increases with flint.

> Who is behind the Flint database? Can we vote on this issue as community?

There has been lots of performance work put into Flint (which was 
originally derived from Quartz), and Olly and I are certainly not going 
to drop Flint.  Since all the tests we've been able to run indicate 
Flint outperforming Quartz, and there are several new features present 
only in Flint, maintaining Quartz is a useless burden on the development 
team (ie, mainly Olly and me), and it will be dropped from the 
distribution in 1.1.

If you want us to improve performance of Flint in any particular cases, 
give us a way to reproduce the performance problems you're seeing 
(ideally, pieces of test code which we can compile and run and which 
will show the performance problems).  Otherwise, we cannot help you, and 
you help nobody.

-- 
Richard