Database left unlocked by Tcl bindings

[Olly Betts]

On Wed, Feb 24, 2016 at 04:30:55PM +0100, Eric J wrote:
> On Wed, 24 Feb 2016 03:17:35 +0000, Olly Betts <olly at survex.com> wrote:
> >On Mon, Feb 22, 2016 at 12:26:27PM +0100, Eric wrote:
> >> % package require xapian 1.0.0
> >> 1.2.18
> > 
> > I've tested with 1.2.18 and can't reproduce this with that version
> > either (is that also the version of xapian-core you're running?  The
> > 1.2.18 above is the bindings version I think).

You didn't answer this...

> > What FS are you running this on?
> 
> ext4

Pretty standard then, and what I tested with.

> > Is use of Tcl actually a factor here, or can you reproduce it with
> > just C++ code?
> > 
> > E.g. using the "simpleindex" example from the xapian-core sources:
> > 
> > examples/simpleindex tmp.db &
> > examples/simpleindex tmp.db
> 
>   lfs at bruno [ /usr/src/sources-deptj/xapian-core-1.2.18 ]$ examples/simpleindex tmp.db &
>   [1] 26157
>   lfs at bruno [ /usr/src/sources-deptj/xapian-core-1.2.18 ]$ examples/simpleindex tmp.db
>   DatabaseLockError: Unable to get write lock on tmp.db: already locked
>   
>   [1]+  Stopped                 examples/simpleindex tmp.db
> 
> so it is presumably not anything to do with the FS or the OS. I am
> hoping that the right Tcl person (whoever that is) may pick something up
> in an strace.

It's clearly not as simple as execl() always releasing the lock, but I
don't think we've ruled out the OS entirely yet - the above isn't
exactly equivalent to the Tcl code, as the two databases are created by
the same process in Tcl but different processes with simpleindex.

Could you try this C++ version:

#include <xapian.h>
int main() {
    Xapian::WritableDatabase db("tmp.db", Xapian::DB_CREATE_OR_OPEN);
    Xapian::WritableDatabase db2("tmp.db", Xapian::DB_CREATE_OR_OPEN);
}

Compile with:

g++ -O2 `xapian-config --cxxflags --libs` doubleopen.cc

And then run:

./a.out

If locking is working, this should fail (and does for me) like so:

terminate called after throwing an instance of 'Xapian::DatabaseLockError'
Aborted

> > More recent Xapian versions will try to use the new OFD locks and avoid
> > the need to fork() and execl(), so will presumably avoid whatever is
> > going on here.  But the OFD locks were added in Linux 3.15, so your
> > kernel isn't quite new enough.
> 
> Yes, I saw that, and it is good, but my chances of moving up soon are
> not good. And I would like to get to the bottom of this anyway.

Indeed - I was noting it more as something to be aware of when testing
newer versions.

Cheers,
    Olly

Cheers,
    Olly