On Sun, Sep 28, 2014 at 11:17 PM, Stefan Sperling <stsp_at_apache.org> wrote:
> On Sun, Sep 28, 2014 at 05:56:01PM -0000, stefan2_at_apache.org wrote:
> > Author: stefan2
> > Date: Sun Sep 28 17:56:01 2014
> > New Revision: 1628093
> >
> > URL: http://svn.apache.org/r1628093
> > Log:
> > Support FSFS format 7 commits in load balanced mixed-architecture
> clusters.
> >
> > At least theoretically, machines with different endianess or off_t sizes
> > might access the same repository on e.g. an iSCSI SAN. Then the machine
> > performing a commit may have a different architecture from the one
> building
> > up the transaction. To allow that, even the intermediate (proto-) index
> > format within those transactions must be platform-independent.
>
> Hi Stefan,
>
> If you describe in some more detail how to set it up I can test this
> scenario for you on OpenBSD with sparc64 (64bit big-endian), amd64
> (64bit little endian), macppc (32bit big-endian) and i386 (32bit little
> endian) machines.
>
Thanks for the offer. So, this is the issue that I addressed
with the above patch:
At the FS API level, transactions can be opened (and implicitly
closed again) many times before being committed. If these
operations are performed from different machines accessing
the same repository, they might not agree upon C struct size,
layout or interpretation. So, for pure API consistency alone,
all data within a txn needs to be portable / architecture independent.
I may be wrong but I vaguely remember that the HTTP client
sends the "commit" operation over a separate connection that
the one it used to build up the txn. If there is a load balancer
in front of the actual servers, that commit may get served by
a different machine.
Testing the general scenario is simpler, though. Have one machine
build up a transaction which touches a few of nodes (such that
different record sizes would result in missing / misaligned data).
Then e.g. copy the repo to a different machine and let that one
do the commit. The result must then pass 'svnadmin verify'.
The critical combinations are little / big endian and 32 bit vs.
64 bit file offsets (not the same thing as 32 / 64 bit in general).
So, 64 bit big endian vs. 32 bit little endian with 32 bit off_t
would probably cover it. Tests with both machines in both
roles (txn creation and txn commit).
-- Stefan^2.
Received on 2014-09-29 19:18:34 CEST