[Xapian-discuss] Tuning Phrase Searching

Olly Betts olly at survex.com
Sat Nov 5 20:18:18 GMT 2005


On Thu, Nov 03, 2005 at 07:32:09AM -0500, tech at dbx.co.uk wrote:
> Is there anything that can be done to speed up phrase searching? It is
> currently a show stopper for our CV search system with queries for common
> terms taking several minutes to execute. Simply ANDing the terms together
> will return in 1-3 seconds.

Even 1-3 seconds for an AND query sounds rather slow.  More RAM could
well help a lot.  What spec is the machine?

And what do slow queries look like?

The flint backend is already noticeably faster for phrase searches (and
other searches) than quartz, so you could try that.  It's "in
development", but the aim is to have it stable in each release, but
with an evolving database format without worrying about compatibility.
The version of flint in 0.9.2 is in use by at least 3 large installations.

Just set environment variable XAPIAN_PREFER_FLINT to a non-empty value
to get Xapian to default to creating flint databases.

For more information see:

http://wiki.xapian.org/FlintBackend

I'm currently writing a new Btree manager which will produce
substantially more compact Btrees.  The differences with the current one
should particularly benefit the positionlist table which has a lot of
keys, many of them fairly long.

> I keep thinking that I must be missing something in either the way I index
> or the way I (or rather the QueryParser) constructs the queries.

Probably not.

Cheers,
    Olly



More information about the Xapian-discuss mailing list