[Xapian-discuss] FLAG_WILDCARD, add_database and performance

[Olly Betts]

On Fri, Aug 01, 2008 at 11:48:57AM +0200, Oliver Flimm wrote:
> I recently started to combine several (around 140) seperate databases
> for a single search request with add_database. I use the xapian perl
> bindings. Additionally I use a match decider to implement facets.

Xapian version?  Platform?

> Using a wildcard in a sequential search results in search times around
> 0.00x to 0.x seconds for each database, but the same search request
> using a combined database handle takes around 200 seconds...

A more comparable test would be against the 140 databases merged into
one.

But it sounds like something is O(n*n) in the number of databases - that
shouldn't be necessary that I can see.

If it's easy to test, see if 100 databases takes about 100 seconds, and
70 about 50 seconds.

> Is there a way to improve request times for the combined search using
> wildcards?

Could you profile to find where the time is spent?  Some tips are here:

http://trac.xapian.org/wiki/ProfilingXapian

Cheers,
    Olly