[Xapian-tickets] [Xapian] #678: Replication of >4GB files fails (was: RemoteDatabase)

[Xapian]

#678: Replication of >4GB files fails
-------------------------+-----------------------------
 Reporter:  matf         |             Owner:  olly
     Type:  defect       |            Status:  assigned
 Priority:  normal       |         Milestone:  1.3.4
Component:  Replication  |           Version:  1.2.20
 Severity:  normal       |        Resolution:
 Keywords:               |        Blocked By:
 Blocking:               |  Operating System:  All
-------------------------+-----------------------------
Changes (by olly):

 * status:  new => assigned
 * component:  Backend-Remote => Replication
 * version:   => 1.2.20
 * milestone:   => 1.3.4


Comment:

 OK, so this is an issue with replication, not the remote backend.  Where
 `sizeof(size_t) == 8` (which is generally will be on 64 bit platforms)
 then changing 28 to 63 looks like the correct fix (28 is the greatest
 multiple of 7 < 32, while 63 is the greatest < 64).  But where
 `sizeof(size_t) == 4` (most 32 bit platforms) this will just cause the
 decoded value to overflow.  I'll sort out a proper fix for this.

 Using the remote backend (or replication) doesn't automatically improve
 query speed as such, but both enable approaches which can improve it.  But
 if you have a single database, simply moving it to be remote will probably
 be slower - there's the extra overhead of serialising data and sending it
 across the network (though you are splitting the load across two machines
 to some extent).

 But with the remote database, you can spread the load of searching a large
 data set by partitioning it by document into N databases and putting each
 on a different server, then searching all N together as remote databases.

 And with replication you can efficiently have copies of a database on many
 servers, which allows a high search load to be split across them.

--
Ticket URL: <http://trac.xapian.org/ticket/678#comment:5>
Xapian <http://xapian.org/>
Xapian