[Xapian-tickets] [Xapian] #757: Testcase qp_scale1 fails consistently

[Xapian]

#757: Testcase qp_scale1 fails consistently
--------------------+-------------------------
 Reporter:  gp1308  |             Owner:  olly
     Type:  defect  |            Status:  new
 Priority:  low     |         Milestone:
Component:  Other   |           Version:
 Severity:  normal  |        Resolution:
 Keywords:          |        Blocked By:
 Blocking:          |  Operating System:  All
--------------------+-------------------------

Comment (by olly):

 It's a somewhat empirical choice (see #308).  In the current version of
 this testcase, "small" times a loop parsing the same query string `q` `n`
 times, and "large" concatenates `n/5` copies of `q` together and parses
 that 5 times.

 We're trying to check that long query strings don't cause scaling
 problems, but the test is not ideal as we incur the overheads of parsing a
 lot more times for the "large" tests.  Ideally the test would be
 constructed such that "small" and "large" would give equal times, and we'd
 just have a margin to allow for random variations due to other loads on
 the machine, etc.  That's how most most of the scaling tests we have work
 (see `tests/harness/scalability.h` which defines `O_N_LOG_N` etc
 thresholds which are each the geometric mean between the scaling
 behaviours we want them to discriminate between - for 10 times the work,
 `O(n)` -> 10, O(log(n)) -> 20, so the threshold is set based on
 `sqrt(10*20)` which is about 14.1).  But this testcase is older that the
 `scalability.h` mechanisms.

 It'd be better to try to adjust this testcase to be more self-balancing
 than just nudge the factor further.

 But I wonder if there's actually a problem here - are you using
 `./configure --enable-log`?  I noticed one of the scaling tests failing
 with that enabled recently, but haven't had a chance to investigate.

 The logging certainly can cause issues - e.g. it can turn a method which
 takes a `Query` object from `O(1)` to `O(n)` by calling
 `Query::get_description()` when it logs the parameter value.

--
Ticket URL: <https://trac.xapian.org/ticket/757#comment:2>
Xapian <https://xapian.org/>
Xapian