[Xapian-discuss] Node.js binding

[Liam]

On Thu, Oct 20, 2011 at 4:33 PM, Richard Boulton <richard at tartarus.org>wrote:

> On 21 October 2011 00:14, Liam <xapian at networkimprov.net> wrote:
> >> Not usually (though for example, a call to an Enquire object will
> >> often result in a call to a Database object inside Xapian, so this can
> >> be true).
> >
> > Wouldn't the user know that the Enquire was bound to that Database
> instance?
>
> Depends on the code structure whether it's easy to know - eg, there
> could be a pool of databases, users might write code which gets a
> database from the pool and makes an enquire object with it, and then
> keeps the enquire object around for a bit after returning the database
> to the pool.
>
> I think this comes down to documenting that an Enquire object will
> make accesses to the Database object it came from for many methods,
> and that such accesses mustn't be performed concurrently.
>

Indeed. We could tie an Enquire to a Database explicitly in the Node
binding; make a Database method which returns an Enquire and keep a list of
them.

>
> Surely you must have many users who allow concurrent connections to xapian
> > databases, so there's some standard way of isolating the connections'
> > activities?
>
> There's no standard way really, but the usual approach is one of:
>
>  - create all Xapian objects needed for a given request from scratch
> each time; don't let any persist, and handle the request in a single
> thread.
>
>  - get some Xapian objects from a pool for each request, but be very
> careful that all derived objects are destroyed before returnng objects
> to the pool.
>
>  - use per thread storage to keep a cache of databases (or whatever
> object is needed), and only use these objects in one thread.  This is
> my preferred approach for searches, generally; the Database.reopen()
> call allows the database version being read to be updated efficiently
> before each request.
>
>  - use a queue and a worker in a thread to perform all access to a
> database.  This is reasonably appropriate for indexing, since only one
> instance of a WritableDatabase for a given path may exist anywhere, so
> modifications need to be serialised to a single thread anyway.
>

So in the Node case, if you were pooling, you'd group any derived objects
with their database. This is a variation of your per-thread storage
approach.

For indexing, you'd maintain a queue and process one entry at a time.