Database left unlocked by Tcl bindings

[Eric J]

On Sat, 27 Feb 2016 19:39:11 +0100 (CET), Eric J <eric at deptj.eu> wrote:
> On Thu, 25 Feb 2016 23:37:52 +0000, Olly Betts <olly at survex.com> wrote:
8>< --------
>> I'm testing with Tcl 8.6 (Debian package 8.6.4+dfsg-3), and it works for
>> me.
>> 
>> So it does seem it must be due to something your Tcl interpreter is
>> doing, but I'm struggling to think what that could be.
>> 
>> If O_CLOEXEC was set on the lock fd when execl() was called, the fd
>> would get closed and the lock released.  But your lsof shows the fd open
>> but not locked in the child process after it has exec-ed cat.
>> 
>> If there were a second fd open on the lock file which gets closed
>> in the child process after the lock is taken, that would release the
>> lock.  But we carefully close all other open fds before taking the
>> lock to avoid that.
> 
> I have tried Tcl 8.6.4 now, and it too has the problem. However with the
> very new Tcl 8.6.5rc2 it works! I still intend to try to find out what
> the problem was, but I can use the 8.5 tclkit for what I was doing when
> this all started, and then move to 8.6.5 when it becomes a real release.

After some more experiments, and some help from the Tcl side, I can now
say that database locks from the Tcl bindings will not function
correctly in the following Tcl versions:

    8.5.18 built with threads (not the default)
    8.6.[1-4] built with threads (default)

but will function correctly in the following Tcl versions:

    8.5.18 built without threads (default)
    8.5.19 built with or without threads
    8.6.[1-4] built without threads (not the default)
    8.6.5 built with or without threads

Earlier 8.5.x are presumably the same as 8.5.18.

This all seems (just my own theory, not proven) to be a collision of
corner cases:

* fork + exec being expected to need to preserve a file lock.
* early creation of a notifier thread expected to be without undesirable
  side-effects

Anyway, the answer is to use Tcl versions as above, or to use
Xapian/kernel combinations where OFD locks are available.

Eric
-- 
ms fnd in a lbry