[Xapian-discuss] Multithreaded read access

[Richard Boulton]

Lucian Nicolescu wrote:
> Inside the xmlrpc server I only open the xapian database for reading once.
> This causes requests to "stay in line" in order to get results and sometimes
> a one operation can take up to 1 second to complete (a time-taking operation
> includes text analisys and some more operations besides the actual Xapian
> interaction) thus delaying all the queued requests.
> 
> I read lots of documents about xapian in mutithreaded environment and
> concluded that even for read access one cannot open the same database more
> than once and perform operations at the same time.

This is a misconception - for read access, you can open the same 
database as many times as you like concurrently (well, as many as your 
operating system will allow - eventually, you'll run out of file 
handles).  However, each of the Database objects you get by opening the 
database must not be accessed concurrently.  A typical model might be to 
create N threads, and use N Database objects, one for each thread.

> I am using Xapian 0.9.6, Python bindings and Twisted Python XMLRPC package.

Just to note - Xapian 0.9.6 is rather old - the latest release is 1.0.2. 
  1.0.2 has the benefit that the Python bindings are set up to release 
the GIL whenever a potentially long-running xapian operation is in 
progress,  which will give better performance in a concurrent environment.

-- 
Richard