[Xapian-devel] Xapian::Database->close() for perl missing

[Websuche :: Felix Ostmann]

2012/4/20 Olly Betts <olly at survex.com>

> On Thu, Apr 19, 2012 at 03:45:36PM +0200, Websuche :: Felix Ostmann wrote:
> > I have a xapian-daemon, which can be queried via http. A
> background-process
> > generated every hour one new index and then remove and create a new
> symlink
> > to the current database.
> >
> > /path/to/index/20120419010000
> > /path/to/index/20120419020000
> > /path/to/index/20120419030000
> > /path/to/index/default => /path/to/index/20120419030000
>
> Note that using symlinks for this isn't recommended - Xapian opens the
> files using the given path, so if it is in the middle of opening the
> database when you switch the symlink, it'll try to open half the old
> one and half the new one, which isn't good at all.
>
> It's better to use a stub database file instead.
>

The problem with the symlink is correct, but will only break 1 request when
the system starts and that also only in theory. If such a open occure, the
next request will close & open again (because the mtime of the iamchert is
now newer).

I cannot follow your meaning of a stub-database, what do you mean?



>
> > So the daemon only check the mtime of /path/to/index/default/iamchert
> befor
> > every request and if it is a new one, he close/reopen the database.
> >
> > The problem is: There is no ->close in perl for a database! So currently
> i
> > override the object. After some days, the prozess have many open
> > filepointers to allready removed databases.
>
> If you destroy all references to the database, it should get closed, so
> this suggests that either the Perl bindings aren't keeping track of the
> references correctly, or there's an object you have around still with a
> reference to the Database (keeping an Enquire or MSet or TermIterator or
> similar will implicitly keep a reference to the database).
>

Perhaps i missed a reference, but i override the database, enquire and
queryparser-object with there new one after a new open. No other objects
should survive there scope.



>
> > I also tried ->reopen, but xapian dont reopen every file, so after
> reopen,
> > i have some files opened from the old database and the record.DB from the
> > new one. Also the filepointer to the old record.DB still exists (i tested
> > with lsof -p $PID).
>
> Calling reopen() will only update you to the latest revision of the
> database you originally opened - it doesn't work through symlinks like
> this (or through stub databases).
>
> > So what is the right way to make a clean shutdown of a opened xapian
> > database?
>
> It should be enough to overwrite the perl variables which hold
> references, e.g.:
>
> $db = $enquire = $mset = undef;
>
> Though it is quite easy to miss an object.  Or maybe there's a missing
> unreference somewhere.
>
> I thought we supported Database::close() in Perl, but it looks like when
> the wrapper was contributed, it was put on WritableDatabase only, which
> was wrong.  You can probably just copy the close bit from
> XS/WritableDatabase.xs to XS/Database.xs if you want to fix it locally.
> If you do, a patch would be great.  Otherwise I'll attend to it when I
> have a bit more spare time.
>

OK, i simple copied the close() from XS/WriteableDatabase.xs to
XS/Database.xs and all works as expected.



>
> Cheers,
>    Olly
>



-- 
Mit freundlichem Gruß
Felix Ostmann

-----------------------------------------------------------
Websuche Search Technology GmbH & Co. KG
Martinistraße 3, D-49080 Osnabrück, Germany
-----------------------------------------------------------
Tel.: +49 (0) 541 / 40666 0, Fax: +49 (0) 541 / 40666 22
Email: info at websuche.com, Web: www.websuche.com
-----------------------------------------------------------
AG Osnabrück - HRA 200252, Ust-IdNr.: DE814737310
-----------------------------------------------------------
Komplementärin: Websuche Search Technology Verwaltungs GmbH
AG Osnabrück - HRB 200359, Geschäftsführer: Ansas Meyer
-----------------------------------------------------------

Die in dieser Email enthaltenen Informationen sind vertrau-
lich zu behandeln und ausschließlich für den Adressaten be-
stimmt. Jegliche Veröffentlichung, Verteilung oder sonstige
in diesem Zusammenhang stehende Handlung  wird ausdrücklich
untersagt.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20120420/6c4d36b2/attachment.htm>