[Xapian-devel] About memory index/search in multithread program

[Yanxiong Lu]

There may be some solutions，for example
class XAPIAN
{
    *static* int InitDatabase();  //for reading only, do not need lock, but
if writing use lock
    int Search(); //safe in one object, do not need lock
};
XIPIAN xp[ THREAD_NUM ];

one thread use one object, they use one database.
these can be in memory with one database.


2014-06-24 20:48 GMT+08:00 Olly Betts <olly at survex.com>:

> On Thu, Jun 19, 2014 at 10:52:13PM +0800, Yanxiong Lu wrote:
> > Why xapian don't support memory  index/search ?
> >
> > I know there is a method can create memory datebase, like this:
> > Xapian::WritableDatabase db(Xapian::InMemory::open());
>
> While the inmemory backend is handy for some things, in its current
> incarnation some operations really aren't very time efficient, and it
> doesn't even try to be very space efficient.
>
> The longer term plan is to replace it, probably sharing code with the
> disk based backends (which would make it fairly easy to do neat things
> like migrating data from "inmemory" to disk).
>
> > *But, if i use these in multithread program,  i need create many
> > datebases!!*
> > Xapian::WritableDatabase db1(Xapian::InMemory::open());  //used in
> thread1
> > Xapian::WritableDatabase db2(Xapian::InMemory::open());  //used in
> thread2
> >
> > because WritableDatabase object isn't thread-safe. And use lock is
> slowly.
>
> Pushing those locks inside the Xapian library won't make things any
> faster - likely just the opposite.  If the API user locks, they
> can choose the granularity, while if Xapian locks, it has to lock
> and unlock around every API method called.
>
> I guess a lock-free, thread-safe inmemory backend implementation might
> be possible, but realistically if you want one any time soon, you're
> probably going to need to contribute most of the code for it.
>
> > *So, is there some solutions that One database, many thread can use??*
>
> The simplest solution currently is probably to create a RAM disk and use
> a disk-based backend in it.
>
> Cheers,
>     Olly
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20140626/d34f7e2c/attachment.html>