[Xapian-discuss] xapian database corrupt with multi-write

[程诚]

I have the same problem. Every five minutes to write the database in my
application,but if the first is not complete, there are mutiple write to a
xapian database in the second time. So the database is currupted
By the way, the xapian verison is 1.2.7. OS is ubuntu10.04

2011/11/2 Olly Betts <olly at survex.com>

> On Tue, Nov 01, 2011 at 10:27:11PM +0800, ????????? wrote:
> > I found sometimes  DatabaseLockError raised but the database is OK. And
> > sometimes database is corrupt.
>
> This is the first report of such an issue I've heard, and we've used the
> same locking approach for a long time, so I think there's something
> particular to your setup which is the key to what is going on.  So if
> you want us to help you, we need to work out what that is, and in order
> to do that, you're going to need to provide precise answers to the
> questions we ask...
>
> > If I run my app by mistake, it is very often that the database will be
> > corrupt. And I am very confused that why the lock file is opened by
> 'cat'.
>
> The lock is held by a "cat" process because fcntl locking has a
> design-flaw which means we need to hold the lock in a subprocess to
> avoid it getting accidentally released, and exec-ing cat avoids dragging
> round pointless VM overhead.  It's really just an implementation detail
> you shouldn't need to worry about.
>
> > > Indeed - what platform (operating system and filesystem) is this on?
> > >
> > Ubuntu 8.04.4
>
> *and filesystem*?
>
> > > Also, what Xapian version?
> > 1.27
>
> There hasn't been a version 1.27.  Since we didn't get to 1.0.27, so I
> guess you mean 1.2.7?
>
> > > > Are you getting the DatabaseLockError raised, and then on subsequent
> > > > open finding the database to be corrupt? Or are you not seeing the
> > > > exception and subsequently finding the database to be corrupt?
> > >
> > > Yes, a precise explanation of what you see would be very helpful too.
> >
> > Sometimes I killed our server, but I found the file is still locked by
> > 'cat'. It is very strange.
> > Maybe the database is corrupt after I start the server again.
>
> Um, that doesn't answer what James asked.
>
> Are you saying that when the second writer tries to get the lock it
> gets DatabaseLockError?  Or that it manages to get the lock despite
> there already being a writer which should have the lock?
>
> If you're able, showing us the *COMPLETE* code to your indexer would
> probably be helpful too.  Just pasting snippets of code isn't much help,
> as we can't try the code then, and you may omit something relevant to
> the problem.
>
> Cheers,
>     Olly
>
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>