[Xapian-tickets] [Xapian] #827: Investigate background fetching of documents from remote

[Xapian]

#827: Investigate background fetching of documents from remote
-----------------------------------+------------------------
        Reporter:  Olly Betts      |      Owner:  Olly Betts
            Type:  enhancement     |     Status:  new
        Priority:  normal          |  Milestone:  2.0.0
       Component:  Backend-Remote  |    Version:
        Severity:  minor           |   Keywords:
      Blocked By:                  |   Blocking:
Operating System:  All             |
-----------------------------------+------------------------
 ''(Spun off #353)''

 The old "network" backend (removed in
 e7c42c964728652432d5530b8f6913bc22bd19c6) implemented specialised
 `request_document()` and `collect_document()` methods. These were used to
 allow the `MSet::fetch()` methods to request the documents which the user
 wanted in bulk, rather than having a request-fetch conversation across the
 link for each document.

 The new "remote" backend which replaced it doesn't have this
 specialisation.  My testing at the time didn't show any speed up, but that
 may not be true in every situation (e.g. higher latency connection, or
 with a cold disk cache).

 Since then, `MSet::fetch()` was hooked up for local backends to trigger a
 pre-fetch request, which can help a lot when the disk cache is cold.

 I've just pushed 02bfcf0541c6beacf25501d95f759baf485b058b which makes
 `MSet::fetch()` send a request to the remote which triggers a prefetch
 there, which should help the case of a remote with a cold disk cache.

 We should prototype and profile to investigate if it would be worthwhile
 implementing something more like the old network backend had.
-- 
Ticket URL: <https://trac.xapian.org/ticket/827>
Xapian <https://xapian.org/>
Xapian