[Xapian-tickets] [Xapian] #743: omindex: delay libmagic checks

[Xapian]

#743: omindex: delay libmagic checks
--------------------+-------------------------------
 Reporter:  olly    |             Owner:  barufa
     Type:  defect  |            Status:  new
 Priority:  normal  |         Milestone:  1.4.x
Component:  Omega   |           Version:  git master
 Severity:  normal  |        Resolution:
 Keywords:          |        Blocked By:
 Blocking:          |  Operating System:  All
--------------------+-------------------------------

Comment (by olly):

 > If the ctime of a file has changed but the mtime is unchanged, we can
 update the existing Document (and avoid re-extracting the text). However,
 I am not sure of which is the cheapest(fastest) way to update it.

 We can, but only when `use_ctime` is set which it isn't the default so
 this is of more limited benefit.  It's also not what this ticket is
 actually about - see #632 for that.

 > Also, I don't know how to test the code properly. At the moment, I am
 using 'make check' but there are others options in the makefile.

 `make check` runs all the tests there are.

 But unfortunately there aren't any tests for this area of omindex
 currently.  There are unit tests for some functionality it uses, but
 nothing for the handling of files.

 > FIXME: except permissions

 Consider the case where we failed to index a file because the user the
 indexer runs as didn't have permission to read it.  Then the permissions
 get fixed, which changes the ctime but not the mtime, but we only check
 the mtime when deciding to retry so won't retry it.  It's really a corner
 case (and probably falls under #632).  We could fix it by just changing to
 checking the ctime to decide whether to retry, but then we'd retry files
 in cases where nothing relevant has actually changed.

 The whole ctime vs mtime thing would be good to address, but it's tricky
 to solve without causing a lot of extra files to be reindexed needlessly -
 probably not something you want to try to tackle as your first bug.

--
Ticket URL: <https://trac.xapian.org/ticket/743#comment:8>
Xapian <https://xapian.org/>
Xapian