[Xapian-tickets] [Xapian] #666: Implement single-file format for glass backend

[Xapian]

#666: Implement single-file format for glass backend
-------------------------+------------------------------
 Reporter:  wgreenberg   |             Owner:  olly
     Type:  enhancement  |            Status:  assigned
 Priority:  normal       |         Milestone:  1.3.x
Component:  Other        |           Version:  SVN trunk
 Severity:  normal       |        Resolution:
 Keywords:               |        Blocked By:
 Blocking:               |  Operating System:  All
-------------------------+------------------------------

Comment (by olly):

 > librt

 That's for timing out check_at_least, which is a new feature added in the
 1.3 development series - if the functions needed aren't present, this
 feature is just disabled.

 > index needs to be "merged" in an other file (ZIM file). Would that be
 possible to have an openIndex() with a baseline offset as additional
 parameter?

 Just `lseek()` the fd to where the Xapian database starts before you pass
 it in.

 > a lot of devices are still not able to deal with files over 4GB

 So you want everything in one file ... which you then split up into
 several files?

 That's not something which the people kindly funding this development
 require (or if they do it wasn't mentioned in the brief), so it's not
 likely to happen as part of my current work on this.

 I had thought the days of the maximum file size being an issue were a
 decade or so behind us, but sadly it seems I was optimistic.  What's the
 actual max file size for these devices, and how big are the databases?
 When the filesize limit was `2GB - 1 byte`, we used to work around it by
 splitting databases into a series of 1GB chunks.  The really annoying
 problem is managing filehandles - for a DB split into a lot of chunks,
 keeping them all open risks running out of filehandles, while having just
 the chunk you're currently reading from open means you can end up opening
 and closing the files a lot.

--
Ticket URL: <http://trac.xapian.org/ticket/666#comment:9>
Xapian <http://xapian.org/>
Xapian