[Xapian-discuss] get_document loop really slow after index update (PHP bindings)

[James Aylett]

On Wed, Oct 21, 2009 at 06:24:04PM +0300, Matei Pavel wrote:

> I'm using the PHP bindings to do a search on about 30-40.000 indexed
> documents that returnds 500-1000 results. The actual search takes
> under 200 miliseconds but the mset iterator loop (that calls
> get_document) takes anywhere from 5 to 21 seconds. This only happens
> for searches made after the index has had some updates.

I'm guessing that on update you're losing enough of the Xapian
database from cache and it needs to be reloaded; and that additionally
something is killing your disk performance. You may be able to probe
more with iostat and vmstat.

> Here is the oprofile callgraph: promotii-reviste.ro/oprofile-callgraph.txt

I'm not great at reading oprofile, but time spent in the Xapian
library itself is around 25k samples. As a comparison, zend_hash_find
seems to have recorded 60k samples. I may be completely misreading
this. (And then there's calls from Xapian down into other libraries,
which if I'm getting things right is perhaps another 30k samples.

J

-- 
  James Aylett

  talktorex.co.uk - xapian.org - uncertaintydivision.org