RE: Testing a shared subversion repository [ netapps partition - NFS
mounted ] with multiple access points [ HOST1, HOST2]. Test consisted of
large parallel commits of the same set of changes .
Results: WEIRD --- a ³dummy² commit with no changes whatsoever [ in one
instance ]
I ran bunch of tests [ using a shared NFS storage ] primarily to determine
if I could write / commit simultaneously from multiple hosts and whether
the locking mechanisms work as advertised.
Configuration used
++++++++++++++++
(a) Shared svn repository located on a netapps partition NFS mounted on
two hosts HOST1, HOST2.
(b) HOST1 had a apache 2.2.x setup with per-fork mpm enabled and with full
permissions to the svn repo.
User access via ³http://URL%b2
© Access to HOST2 using svn+ssh protocol also with full write access.
User access via ³svn+ssh://URL²
(d) Locking enabled on netapps + nfs hard mount [ default]
TEST
++++
1: I made a number of commits [ both small and big] using svn+ssh and http
protocols from both HOST1 and HOST2 . Results were OK
2: Then I made two large simultaneous commits after deleting a very large
directory. Please note that I was deleting the same set of files -- one
based off HOST1 and the other one based off HOST2 [ http and svn+ssh
protocols ]
Example:
svn delete my-WC-HOST1/html/intern
svn delete my-WC-HOST2/html/intern
svn commit m ³Delete html/intern² my-WC-HOST1 [ uses http://URL ]
svn commit m ³Delete html/intern² my-WC-HOST2 [ uses svn+ssh://URL ]
RESULTS
++++++++
What I would have expected to see is the commit going through from one HOST
and a error / failure / conflict from the 2nd HOST since the same set of
files have already been deleted.
However the results were unexpected --- the commit from HOST1 went through
OK ‹ strangely the commit from HOST2 also went thorough. However the second
commit log indicated *NO* changes -- a completely ³dummy² commit generated
via HOST2.
$ svn log -v -r 123386 http://HOST1/svnroot_nfs
==>>
------------------------------------------------------------------------
r123386 | mvaradachari | 2008-09-25 17:39:40 -0700 (Thu, 25 Sep 2008) | 1
line
Deleting html/intern
==>>
$ svn log -v -r 123385 http://HOST1/svnroot_nfs
==>>
------------------------------------------------------------------------
r123385 | mvaradachari | 2008-09-25 17:39:39 -0700 (Thu, 25 Sep 2008) | 1
line
Changed paths:
D /tfb/branches/mvaradachari_b1/html/intern
Deleting html/intern
==>>
Q: Has anyone seen this type of behaviour before ?? The log of 123386
indicates no change.
The output of rev# 123386 from /svnroot_nfs/db/revs/123/123386
Cat /svnroot_nfs/db/revs/123/123386
===>>>>
id: 0.0.r123386/0
type: dir
pred: 0.0.r123385/29588
count: 123386
text: 123385 28688 887 887 1b234e9d9466d77921f61ebfaeb9bdd8
cpath: /
copyroot: 0 /
is-fresh-txn-root: y
minfo-cnt: 184
0 186
====>>
The output of rev# 123385 from /svnroot_nfs/db/revs/123/123385
cat /svnroot_nfs/db/revs/123/123385
===>>>
........
........
..........
id: 0.0.r123385/29588
type: dir
pred: 0.0.r123384/0
count: 123385
text: 123385 28688 887 887 1b234e9d9466d77921f61ebfaeb9bdd8
cpath: /
copyroot: 0 /
minfo-cnt: 184
8t-11.0-5930.r123372/55768 delete false false
/tfb/branches/mvaradachari_b1/html/intern
===>>>
Is this a bug ? Or did I miss something.
Cheers
murli
On 9/23/08 7:59 AM, "david.x.grierson_at_jpmorgan.com"
<david.x.grierson_at_jpmorgan.com> wrote:
> Hi,
>
> We ran into problems with Subversion 1.4.5 using the worker MPM because
> the apr-util library which controls the DAV activities database uses DBM
> files to track changes.
>
> DBM files require POSIX locks to be able to be held on files - since
> threads aren't real processes they don't obey POSIX locking properly and
> so corruption to your repository can occur - particular when large commits
> are taking place.
>
> Lieven Govaerts pointed out to me on the "Shared storage for subversion
> repository" thread that "Subversion 1.5 has its own file-based mechanism
> for maintaining the DAV activity mappings, so the above is no longer
> true." So it could well be that the worker MPM may now be feasible.
>
> We're consuming 1TB of storage across 1056 repositories which are
> receiving approx 3,500 commits per day. These are being hosted by a single
> headed Apache host which has 2 x dual-core Xeon 2.66GHz and 16Gb of RAM.
> The NFS data is held on a FAS3050 NetApp filer which is replicated to a DR
> site via NetApp SnapMirror software. There is identical hardware in both
> production and DR with identical software and configuration.
>
> Dg.
> --
> David Grierson
> JPMorgan - IB Architecture - Source Code Management Consultant
> GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson_at_jpmorgan.com
> Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
>
>
>
>
> Murli Varadachari <mvaradachari_at_facebook.com>
> 23/09/2008 15:38
>
> To
> Steve Whitson <steven.whitson_at_gmail.com>, <users_at_subversion.tigris.org>
> cc
> Internal Use <mvaradachari_at_facebook.com>
> Subject
> Re: subversion on NFS
>
>
>
>
>
>
>
> I have just set up a shared subversion repository [ on a NFS partition ]
> and am in the process of testing simultaneous read-writes from a number of
> hosts. SO far the tests seem ok. The current setup uses svn+ssh
> protocols.
>
> Down the road I intend to use both http(s) and svn protocols.
>
> I am curious why you ran into corruption problems using ³worker mpm² -- is
> that still a problem with svn 1.5. [ on a fedora core linux ]
>
> Also you mentioned that you are using about 1TB of repository storage ‹ is
> that a single repository.??
>
> Cheers
> murli
>
>
> On 9/23/08 7:19 AM, "Steve Whitson" <steven.whitson_at_gmail.com> wrote:
>
> Thanks David, I appreciate your feedback!
>
> david.x.grierson_at_jpmorgan.com wrote:
>> > Hi,
>> >
>> > We successfully run our services from NFS stored repositories using the
>> > Apache prefork MPM serving over 1TB of repository storage transferring
> an
>> > average of over 60GB of traffic per day (1920.60Gb of data transferred
> for
>> > August).
>> >
>> > When I upgraded to 1.4.5 early this year I ran into repository
> corruption
>> > problems - these turned out to be because I had switched over to using
> the
>> > worker MPM.
>> >
>> > Regards,
>> >
>> > Dg.
>> > --
>> > David Grierson
>> > JPMorgan - IB Architecture - Source Code Management Consultant
>> > GDP 228-5574 / DDI +44 141 228 5574 / Email
> david.x.grierson_at_jpmorgan.com
>> > Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
>> >
>> >
>> >
>> >
>> > Steve Whitson <steven.whitson_at_gmail.com>
>> > 22/09/2008 20:26
>> >
>> > To
>> > "users_at_subversion.tigris.org" <users_at_subversion.tigris.org>
>> > cc
>> >
>> > Subject
>> > subversion on NFS
>> >
>> >
>> >
>> >
>> >
>> >
>> > When serving subversion (1.5.2 fsfs) through apache (2.2.9)... (this is
>> > on a Solaris 10 system with nfs from netapp).
>> >
>> > I see where subversion should not be used with the apache worker MPM
>> > module. Is using the prefork MPM module ok?
>> >
>> > Another question I have, can I allow svnserve access to my set of
>> > repositories (read/write) while at the same time serving them up through
>> > apache (something I'm toying with) or is doing this going to eventually
>> > cause corruption?
>> >
>> > I've read the FAQ on NFS, and several past threads:
>> > http://subversion.tigris.org/faq.html#nfs
>> > http://svn.haxx.se/users/archive-2008-05/0048.shtml
>> > http://svn.haxx.se/dev/archive-2007-04/0288.shtml
>> >
>> > Thanks much,
>> > -Steve
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
>> > For additional commands, e-mail: users-help_at_subversion.tigris.org
>> >
>> >
>> >
>> >
>> > Generally, this communication is for informational purposes only
>> > and it is not intended as an offer or solicitation for the purchase
>> > or sale of any financial instrument or as an official confirmation
>> > of any transaction. In the event you are receiving the offering
>> > materials attached below related to your interest in hedge funds or
>> > private equity, this communication may be intended as an offer or
>> > solicitation for the purchase or sale of such fund(s). All market
>> > prices, data and other information are not warranted as to
>> > completeness or accuracy and are subject to change without notice.
>> > Any comments or statements made herein do not necessarily reflect
>> > those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>> >
>> > This transmission may contain information that is privileged,
>> > confidential, legally privileged, and/or exempt from disclosure
>> > under applicable law. If you are not the intended recipient, you
>> > are hereby notified that any disclosure, copying, distribution, or
>> > use of the information contained herein (including any reliance
>> > thereon) is STRICTLY PROHIBITED. Although this transmission and any
>> > attachments are believed to be free of any virus or other defect
>> > that might affect any computer system into which it is received and
>> > opened, it is the responsibility of the recipient to ensure that it
>> > is virus free and no responsibility is accepted by JPMorgan Chase &
>> > Co., its subsidiaries and affiliates, as applicable, for any loss
>> > or damage arising in any way from its use. If you received this
>> > transmission in error, please immediately contact the sender and
>> > destroy the material in its entirety, whether in electronic or hard
>> > copy format. Thank you.
>> > Please refer to http://www.jpmorgan.com/pages/disclosures for
>> > disclosures relating to UK legal entities.
>> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
> For additional commands, e-mail: users-help_at_subversion.tigris.org
>
>
>
>
> Generally, this communication is for informational purposes only
> and it is not intended as an offer or solicitation for the purchase
> or sale of any financial instrument or as an official confirmation
> of any transaction. In the event you are receiving the offering
> materials attached below related to your interest in hedge funds or
> private equity, this communication may be intended as an offer or
> solicitation for the purchase or sale of such fund(s). All market
> prices, data and other information are not warranted as to
> completeness or accuracy and are subject to change without notice.
> Any comments or statements made herein do not necessarily reflect
> those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to UK legal entities.
>
Received on 2008-09-29 02:18:40 CEST