[Xapian-tickets] [Xapian] #337: Extend test of behaviour of databases after close
nobody at xapian.org
Tue Apr 17 22:24:12 BST 2018
#337: Extend test of behaviour of databases after close
Reporter: richard | Owner: gp1308
Type: defect | Status: new
Priority: low | Milestone: 1.4.6
Component: Test Suite | Version: SVN trunk
Severity: normal | Resolution:
Keywords: | Blocked By:
Blocking: | Operating System: All
Comment (by olly):
I'm not sure the local database case is as interesting as the remote one,
as just closing the file descriptors isn't a realistic simulation of an
actual failure mode, but FWIW it'd really need to happen in the test
harness code, not the backend code. The `Database::Internal` object (e.g.
`GlassDatabase`) knows what fds are open on the database, but there's no
way to ask it to report that information, and it doesn't seem an
appropriate public API. And better to avoid adding special private APIs
just for the test harness if we can, not least because it's har
The test harness supports running tests on different "backends" (which are
slightly different to the library's concept of a backend, since a test
harness "backend" specifies the whole setup - e.g. the test harness backed
"remotetcp_glass" is the remote library-backend talking TCP to a remove
server using the glass library-backend for its database).
The harness knows the backend it's running tests for, and what features
that supports (e.g. spelling, transactions) and testcases are flagged with
what feature combinations they need. The testcase just tells the harness
things like "make me a `WritableDatabase` filled with data by generator
function X", and the harness creates one of the right type with that data
So the harness could (at least on platforms with `/proc/self/fd`) note
which fds are in use, create the database, and check which additional fds
are now in use - those are presumably open on the database we just opened.
One potential flaw is that some library call made during DB
opening/creation might open some file used internally lazily - e.g.
`/dev/urandom` might get opened to generate a UUID for the database. If
that happens, we'd assume that file was one for the database. It's also
hard to make work if the test harness becomes multi-threaded, which is
something that will probably happen in the future to speed up running the
Or perhaps compare the targets of the symlinks in `/proc/self/fd` with the
directory path of the database, though that potentially has path
normalisation issues, and I'm not sure if all platforms with
`/proc/self/fd` have symlinks there (Linux does at least).
There's existing code in `xapian-core/tests/harness/fdtracker.cc` which
tracks fds so it can report when a testcase fails to close all the file
descriptors it opened. This works on Linux and a few other platforms
which have `/proc/self/fd` - on other platforms we'll fail to open that
special directory (which contains an entry for each fd open by the current
process) and the FD checking is disabled automatically when that happens.
I notice it has a subtle bug actually - the `DIR` handle that `opendir()`
returns wraps a file descriptor, so that'll be reported by `readdir()` -
you can see this effect if you `ls /proc/self/fd`:
$ ls -l /proc/self/fd
lrwx------ 1 olly olly 64 Apr 18 09:16 0 -> /dev/pts/8
lrwx------ 1 olly olly 64 Apr 18 09:16 1 -> /dev/pts/8
lrwx------ 1 olly olly 64 Apr 18 09:16 2 -> /dev/pts/8
lr-x------ 1 olly olly 64 Apr 18 09:16 3 -> /proc/20289/fd
The first three entries are stdin, stdout, stderr all pointing to the
terminal, the last is open on the directory `/proc/self/fd` symlinks to
and is from `ls` calling `opendir()` (or perhaps actually some lower level
system call which works in a similar way).
So really the FD tracking should get that fd using `dirfd()` on the `DIR`
handle and discount it.
Ticket URL: <https://trac.xapian.org/ticket/337#comment:22>
More information about the Xapian-tickets