[Xapian-discuss] get_data not fast enough for query matches

[James Aylett]

On Thu, Feb 02, 2006 at 03:00:54PM +0000, Salem Berhanu wrote:

> I am not storing anything in the document data other than the
> ids. Eventually I will link to an external database but I will do it
> in ranges of not more than 50 at a time. However, I will need the
> initial compete ids to combine with results from other xapian
> dbs. This is because each document is broken up into chunks (since
> the information can be logically divided) and indexed in separate
> dbs. (eg. there is a title db, a description db ... ) I want to be
> able to combine the results across these dbs using boolean
> expressions (since I am assuming there isn't a built in way of doing
> this).

I'm sorry, there's not much point in trying to answer questions like
this without understand what you're actually trying to achieve. I
don't know why you're using several different databases, for
instance.

Let's go back to the beginning. What sort of performance issues are
you actually seeing? Have you investigated performance using the
standard performance tools for your platform? For instance, are you
running out of file buffer cache at the point that you start accessing
the document data? Other people have handled larger data sets that
you, so it may be that your hardware or configuration isn't quite
right for what you're trying to do.

(And please keep the discussion on list where others can help you as
well :-)

Cheers,
James

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james at tartarus.org                               uncertaintydivision.org