[Xapian-tickets] [Xapian] #823: Unit test ioblock1 failure

[Xapian]

#823: Unit test ioblock1 failure
--------------------------+-------------------------------
 Reporter:  someplaceguy  |             Owner:  Olly Betts
     Type:  defect        |            Status:  new
 Priority:  normal        |         Milestone:  1.4.25
Component:  Test Suite    |           Version:  1.4.24
 Severity:  normal        |        Resolution:
 Keywords:                |        Blocked By:
 Blocking:                |  Operating System:  Linux
--------------------------+-------------------------------
Comment (by someplaceguy):

 > Please can you try the attached patch on your compressed ZFS setup.

 I can confirm that the test now writes random data to the file, but
 unfortunately the test still fails:

 {{{
 $ rm .unittest_ioutils1; ./runtest ./unittest -v ioblock1; stat
 .unittest_ioutils1; sleep 5; echo; stat .unittest_ioutils1

 Running test: ioblock1... FAILED
 unittest.cc:955: (statbuf.st_blocks) >= (BLOCK_SIZE / 512 * 2)
 Evaluates to: 1 >= 4


   File: .unittest_ioutils1
   Size: 13312           Blocks: 1          IO Block: 13312  regular file
 Device: 0,30    Inode: 947166      Links: 1
 Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
 Access: 2023-12-20 16:36:35.801828622 +0000
 Modify: 2023-12-20 16:36:35.810828709 +0000
 Change: 2023-12-20 16:36:35.810828709 +0000
  Birth: 2023-12-20 16:36:35.801828622 +0000

   File: .unittest_ioutils1
   Size: 13312           Blocks: 9          IO Block: 13312  regular file
 Device: 0,30    Inode: 947166      Links: 1
 Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
 Access: 2023-12-20 16:36:35.801828622 +0000
 Modify: 2023-12-20 16:36:35.810828709 +0000
 Change: 2023-12-20 16:36:35.810828709 +0000
  Birth: 2023-12-20 16:36:35.801828622 +0000
 }}}

 The issue seems to be that st_blocks is still 1 when the unit test fails.
 However, as you can see, after waiting a few seconds, st_blocks becomes 9.

 At first, I suspected that the unit test wasn't calling fsync() or
 fdatasync(), but in fact, it is calling fdatasync():

 {{{
 $ strace ./runtest ./unittest -v ioblock1

 (...)

 openat(AT_FDCWD, ".unittest_ioutils1", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC,
 0666) = 4
 pwrite64(4,
 "_\313\363.\273\307\231\230'r'\31\16u\335U\231$\265\246\5\16\370\270\325\3606\0.\376.\236"...,
 1024, 1024) = 1024
 pread64(4,
 "_\313\363.\273\307\231\230'r'\31\16u\335U\231$\265\246\5\16\370\270\325\3606\0.\376.\236"...,
 1024, 1024) = 1024
 fdatasync(4)                            = 0
 pwrite64(4,
 "\356\340\335\215\v*\6i`\307\317z\374\374&`\230\277ad\370\324\327\221\326\20w\tj\"E\3"...,
 1024, 12288) = 1024
 pread64(4,
 "\356\340\335\215\v*\6i`\307\317z\374\374&`\230\277ad\370\324\327\221\326\20w\tj\"E\3"...,
 1024, 12288) = 1024
 fdatasync(4)                            = 0
 newfstatat(4, "", {st_mode=S_IFREG|0644, st_size=13312, ...},
 AT_EMPTY_PATH) = 0
 futex(0x7f929e9ac070, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 write(1, " \33[1m\33[31mFAILED\33[0m\n", 21 FAILED
 ) = 21
 write(1, "unittest.cc:955: (statbuf.st_blo"..., 85unittest.cc:955:
 (statbuf.st_blocks) >= (BLOCK_SIZE / 512 * 2)
 Evaluates to: 1 >= 4

 ) = 85
 }}}

 I've looked to see if this could be a bug in ZFS and I found this ZFS
 issue:

 https://github.com/openzfs/zfs/issues/13991

 Quoting from a comment there from a ZFS developer:

 > (...) fsync() makes no difference here, because it doesn't commit the
 transaction, but rather just ensures the change is on the journal (ZIL) so
 it can be replayed in a crash.
 >
 > I don't know if this can be really called a bug. POSIX and the various
 manpages all seem say "number of blocks allocated", which I can argue is
 technically correct (ZFS doesn't allocate anything until it actually comes
 time to write things).
 >
 > (...) So my inclination is to say "not a bug". But, if you've got a
 program that reasonably relies on st_blocks being instantly accurate, or a
 better source of how this should be required to work, please reply with
 details.
-- 
Ticket URL: <https://trac.xapian.org/ticket/823#comment:3>
Xapian <https://xapian.org/>
Xapian