On 30.09.2014 11:15, Stefan Sperling wrote:
> On Mon, Sep 29, 2014 at 07:18:05PM +0200, Stefan Fuhrmann wrote:
>> On Sun, Sep 28, 2014 at 11:17 PM, Stefan Sperling <stsp_at_apache.org> wrote:
>>
>>> On Sun, Sep 28, 2014 at 05:56:01PM -0000, stefan2_at_apache.org wrote:
>>>> Author: stefan2
>>>> Date: Sun Sep 28 17:56:01 2014
>>>> New Revision: 1628093
>>>>
>>>> URL: http://svn.apache.org/r1628093
>>>> Log:
>>>> Support FSFS format 7 commits in load balanced mixed-architecture
>>> clusters.
>>>> At least theoretically, machines with different endianess or off_t sizes
>>>> might access the same repository on e.g. an iSCSI SAN. Then the machine
>>>> performing a commit may have a different architecture from the one
>>> building
>>>> up the transaction. To allow that, even the intermediate (proto-) index
>>>> format within those transactions must be platform-independent.
>>> Hi Stefan,
>>>
>>> If you describe in some more detail how to set it up I can test this
>>> scenario for you on OpenBSD with sparc64 (64bit big-endian), amd64
>>> (64bit little endian), macppc (32bit big-endian) and i386 (32bit little
>>> endian) machines.
>>>
>> Thanks for the offer. So, this is the issue that I addressed
>> with the above patch:
>>
>> At the FS API level, transactions can be opened (and implicitly
>> closed again) many times before being committed. If these
>> operations are performed from different machines accessing
>> the same repository, they might not agree upon C struct size,
>> layout or interpretation. So, for pure API consistency alone,
>> all data within a txn needs to be portable / architecture independent.
>>
>> I may be wrong but I vaguely remember that the HTTP client
>> sends the "commit" operation over a separate connection that
>> the one it used to build up the txn. If there is a load balancer
>> in front of the actual servers, that commit may get served by
>> a different machine.
>>
>> Testing the general scenario is simpler, though. Have one machine
>> build up a transaction which touches a few of nodes (such that
>> different record sizes would result in missing / misaligned data).
>> Then e.g. copy the repo to a different machine and let that one
>> do the commit. The result must then pass 'svnadmin verify'.
>>
>> The critical combinations are little / big endian and 32 bit vs.
>> 64 bit file offsets (not the same thing as 32 / 64 bit in general).
>> So, 64 bit big endian vs. 32 bit little endian with 32 bit off_t
>> would probably cover it. Tests with both machines in both
>> roles (txn creation and txn commit).
>>
>> -- Stefan^2.
> How do you open a transaction and postpone the commit?
> Using some custom code written against the FS API?
> Or can some tool such as svnmucc already do this?
I think you have to write a tool that works agains the repos API. The
Python bindings should be good enough for that.
> I presume you rely on apr_off_t, not off_t, right? I.e. it's
> enough to recompile APR and SVN with or without large file
> support to switch between 32bit and 64bit apr_off_t?
AFAIR changing largefile support does not change the APR ABI since 1.0.
-- Brane
Received on 2014-09-30 11:21:32 CEST