#708: Implement database fixing for glass
---------------------------+-------------------------------
Reporter: olly | Owner: olly
Type: defect | Status: new
Priority: normal | Milestone: 1.4.x
Component: Backend-Glass | Version: git master
Severity: normal | Resolution:
Keywords: | Blocked By:
Blocking: | Operating System: All
---------------------------+-------------------------------
Description changed by olly:
Old description:
> For chert, we have the `DBCHECK_FIX` mode for database checking (which
> you can activate from `xapian-check` by passing `F` in the flags) which
> will attempt to fix some problems:
>
> * Regenerate `iamchert` if not valid
> * Regenerate `.baseA`/`.baseB` files from `.DB` files
>
> While glass should be less susceptible to small, frequently-replaced
> files getting truncated or otherwise corrupted (because it no longer has
> the `.baseA` or `.baseB` files, we ought to have the equivalent support
> for glass - that's probably means regenerating `iamglass` if not valid.
>
> The complex part of this is fixing up the freelists, which are now stored
> in unused blocks in the database. That requires writing to the table
> file, which is potentially unwise if there's been damage, as it could
> make things worse. The simplest approach would be to set up empty
> freelists (which would potentially mean lost space that couldn't be
> reclaimed in normal use). Then you'd have a database which could read
> and compacted.
>
> If we implement regenerating missing freelists, that could easily be
> triggered if the freelists don't match the blocks in use.
New description:
For chert, we have the `DBCHECK_FIX` mode for database checking (which you
can activate from `xapian-check` by passing `F` in the flags) which will
attempt to fix some problems:
* Regenerate `iamchert` if not valid
* Regenerate `.baseA`/`.baseB` files from `.DB` files
While glass should be less susceptible to small, frequently-replaced files
getting truncated or otherwise corrupted (because it no longer has the
`.baseA` or `.baseB` files) we ought to have the equivalent support for
glass - that's probably means regenerating `iamglass` if not valid.
The complex part of this is fixing up the freelists, which are now stored
in unused blocks in the database. That requires writing to the table
file, which is potentially unwise if there's been damage, as it could make
things worse. The simplest approach would be to set up empty freelists
(which would potentially mean lost space that couldn't be reclaimed in
normal use). Then you'd have a database which could read and compacted.
If we implement regenerating missing freelists, that could easily be
triggered if the freelists don't match the blocks in use.
--
--
Ticket URL: <https://trac.xapian.org/ticket/708#comment:2>
Xapian <https://xapian.org/>
Xapian