[Xapian-discuss] multiple database update scripts

[Jim Spath]

Richard Boulton wrote:
> On Tue, Mar 17, 2009 at 11:19:34AM -0400, Jim Spath wrote:
>> We recently split the updating of a Xapian database into two separate 
>> processes.
>>
>> Process A selects items which need reindexing, writes a datafile, and 
>> runs scriptindex against this file to update the database.
>>
>> Process B gatherers some useful meta for all items and updates each 
>> item's data in the Xapian database using Search::Xapian.
>>
>> When process B is running and process A tries to run, scriptindex spits 
>> out many of these:
>>
>> Database locked ... retrying
>> Database locked ... retrying
>> Database locked ... retrying
>>
>> And each time it does so, a zombie process is spawned.  Left unchecked, 
>> this can spawn hundreds of processes.  Is this the expected behavior for 
>> scriptindex when it can't get a database lock?
> 
> The "database locked" messages are expected.  Only one writer is allowed on
> a Xapian database at a time, so one process will block the other.
> 
> It shouldn't be leaving zombie processes, though.
> 
>> My main question is if its possible to have two processes trying to 
>> update the Xapian database at the same time?
> 
> No, it is not possible.

Hi Richard,

I figured as much ... I'll just run the processes sequentially then.

As far as the zombie processes are concerned, here's some information 
about our environment:

xapian-omega 1.0.7
Ubuntu 6.06.2 LTS x_64 Server Edition

Thanks!
- Jim