[Xapian-tickets] [Xapian] #767: Make testcases using different backends run in parallel

[Xapian]

#767: Make testcases using different backends run in parallel
------------------------------+-------------------------------------
 Reporter:  Guruprasad Hegde  |             Owner:  Guruprasad Hegde
     Type:  enhancement       |            Status:  new
 Priority:  normal            |         Milestone:
Component:  Test Suite        |           Version:  git master
 Severity:  normal            |        Resolution:
 Keywords:                    |        Blocked By:
 Blocking:                    |  Operating System:  All
------------------------------+-------------------------------------
Comment (by Olly Betts):

 https://github.com/xapian/xapian/pull/210 has the prototype using
 automake's parallel test harness.

 I tried updating that (and merging honey into the list of glass-based
 testsuite backends since we compact the glass DB to give the honey one)
 and the speed up is disappointing (~20% IIRC), I think mostly because the
 glass-based list is most of the work.  I didn't copy the stats off my
 laptop but I'll try to remember to add them next time I turn it on.

 I have come up with a cheap way to schedule though.

 If we annotate testcases (could be automatically derived) with the
 database names they use then we can partition the testcases such that any
 which use the same DB are in the same partition.  Some use more than one,
 and that will kind of span between DBs and pull them into the same
 partition, but many use their own DB or share a DB but without such
 overlaps.

 We order these partitions by decreasing expected time to process (could be
 just by number of testcases as a simple approximation, but we could
 feedback from actual runtime), then each worker subprocess just gets the
 next partition from the list to work through when it needs more to do.
 This simple greedy algorithm should work well as we have a load of small
 partitions which should help even out the end of the run between workers.

 These partitions can take into account testsuite backends, overlap between
 them (e.g. glass and honey) and which testcases run for each.

 If we aren't running in parallel we could even run testcases in the same
 order as currently by having a suitable alternative partition set for
 that.
-- 
Ticket URL: <https://trac.xapian.org/ticket/767#comment:9>
Xapian <https://xapian.org/>
Xapian