xapian 1.4 performance issue

[Olly Betts]

On Fri, Dec 08, 2017 at 11:08:00AM +0100, Jean-Francois Dockes wrote:
> This is the only really short term solution: any other is weeks or months
> away. Is the "stub database" feature the appropriate way to create Chert
> databases with Xapian 1.4 ?

With 1.4 you can pass Xapian::DB_BACKEND_CHERT in the flags when
constructing the WritableDatabase object.

I noticed recently that this doesn't quite work as advertised in the
case when the database already exists but is not of the specified type.
It's meant to just open the database in that case (and ignore the
backend hint), but it actually seems to create a new database with the
specified backend in the same directory.  I'll fix that, but obviously
that won't help with existing releases.  You can try with
Xapian::DB_OPEN first, then Xapian::DB_BACKEND_CHERT if that fails,
though that's slightly racy.  Not sure there's a better workaround
though.

> Another possibility for me would be to decide that Chert is good enough and
> appropriate for Recoll, and bundle it together with the appropriate Xapian
> parts.

That wouldn't be popular with distros packaging recoll - they'll want to
use their existing Xapian packages instead of a bundled code copy, e.g.
see:

https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_party_code
https://fedoraproject.org/wiki/Bundled_Libraries
https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies#When_code_is_bundled.3F

It also means you wouldn't benefit from improvements in new Xapian
releases, and would end up having to maintain the old version you picked
yourself.

Cheers,
    Olly