[Xapian-tickets] [Xapian] #651: Avoid low fds

[Xapian]

#651: Avoid low fds
---------------------------+-----------------------------
 Reporter:  olly           |             Owner:  olly
     Type:  enhancement    |            Status:  assigned
 Priority:  normal         |         Milestone:  1.3.3
Component:  Backend-Glass  |           Version:
 Severity:  normal         |        Resolution:
 Keywords:                 |        Blocked By:
 Blocking:                 |  Operating System:  All
---------------------------+-----------------------------
Changes (by olly):

 * status:  new => assigned


Comment:

 This code sets the file position to the maximum possible {{{off_t}}}
 value, which seems to work well at least on x86 Linux 2.6.32-68-generic
 (attempts to write a single byte with {{{write()}}} fail with {{{EFBIG}}})
 and x86_64 Linux 3.2.0-4-amd64 (fails with {{{EINVAL}}}):

 {{{
 #define SHIFT (sizeof(off_t) * 8 - 2)
 lseek(fd, (((off_t)1 << SHIFT)-1)|((off_t)1 << SHIFT), SEEK_SET);
 }}}

 Meanwhile {{{pwrite()}}} works correctly.

 If a system's libc emulates {{{pwrite()}}} with {{{lseek()}}} followed by
 {{{write()}}} then this initial {{{lseek()}}} will be harmless, but we
 won't be protected from stray writes.

 Theoretically, we could protect ourselves by opening the file with
 {{{O_APPEND}}} which shouldn't affect {{{pwrite()}}}.  However on Linux it
 does, and even where this worked it would result in extra garbage
 appearing in the file, whereas setting the offset absurdly high makes
 stray writes fail.  The only downside I can see is that if some system did
 support such a large offset, the file size would increase to a ludicrous
 size.  But most Unix FSes don't explicitly store all the zero-filled
 blocks this would create, so it wouldn't generally fill up huge amounts of
 disk, and the huge file size is a clear clue that there's a stray write
 somewhere.

 This technique really needs testing on more systems - I'll tidy up the
 test program to more clearly report the outcome and attach it here.

--
Ticket URL: <http://trac.xapian.org/ticket/651#comment:3>
Xapian <http://xapian.org/>
Xapian