[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: subversion on NFS

From: Murli Varadachari <mvaradachari_at_facebook.com>
Date: Mon, 29 Sep 2008 07:49:22 -0700

Please open a separate thread -- the problem you are having in not related
to this.

murli

On 9/29/08 12:17 AM, "Rafiuddin M K" <rafiuddin.mk_at_gmail.com> wrote:

>
> I am really new to SVN and Have a problem. Repeatedly, My Server where SVN is
> Installed Group is getting read only, though it having and showing WRITE/READ
> Permissions. Initially thought due to running of some Dump Scripts. But the
> problem resist after stopping that also.
>
> can anyone suggest on this.....
>
> Error: Commit failed (details follow):
> Error: Can't create directory
> '/group/REPOSITORY/project/db/transactions/1578-1.txn': Read-only file system
>

On Mon, Sep 29, 2008 at 6:47 AM, Murli Varadachari
<mvaradachari_at_facebook.com> wrote:
>
> RE: Testing a shared subversion repository [ netapps partition - NFS mounted
> ] with multiple access points [ HOST1, HOST2]. Test consisted of large
> parallel commits of the same set of changes .
>
> Results: WEIRD --- a "dummy" commit with no changes whatsoever [ in one
> instance ]
>
> I ran bunch of tests [ using a shared NFS storage ] primarily to determine if
> I could write / commit simultaneously from multiple hosts and whether the
> locking mechanisms work as advertised.
>
> Configuration used
> ++++++++++++++++
>
> (a) Shared svn repository located on a netapps partition ­ NFS mounted on two
> hosts HOST1, HOST2.
>
> (b) HOST1 had a apache 2.2.x setup with per-fork mpm enabled and with full
> permissions to the svn repo.
> User access via "http://URL"
>
> © Access to HOST2 using svn+ssh protocol also with full write access.
> User access via "svn+ssh://URL"
>
> (d) Locking enabled on netapps + nfs hard mount [ default]
>
> TEST
> ++++
>
> 1: I made a number of commits [ both small and big] using svn+ssh and http
> protocols from both HOST1 and HOST2 . Results were OK
>
> 2: Then I made two large simultaneous commits after deleting a very large
> directory. Please note that I was deleting the same set of files -- one based
> off HOST1 and the other one based off HOST2 [ http and svn+ssh protocols ]
>
> Example:
>
> svn delete my-WC-HOST1/html/intern
>
> svn delete my-WC-HOST2/html/intern
>
> svn commit ­m "Delete html/intern" my-WC-HOST1 [ uses http://URL ]
>
> svn commit ­m "Delete html/intern" my-WC-HOST2 [ uses svn+ssh://URL ]
>
>
> RESULTS
> ++++++++
>
> What I would have expected to see is the commit going through from one HOST
> and a error / failure / conflict from the 2nd HOST since the same set of files
> have already been deleted.
>
> However the results were unexpected --- the commit from HOST1 went through OK
> ‹ strangely the commit from HOST2 also went thorough. However the second
> commit log indicated *NO* changes -- a completely "dummy" commit generated
> via HOST2.
>
> $ svn log -v -r 123386 http://HOST1/svnroot_nfs
>
> ==>>
> ------------------------------------------------------------------------
> r123386 | mvaradachari | 2008-09-25 17:39:40 -0700 (Thu, 25 Sep 2008) | 1 line
>
> Deleting html/intern
>
> ==>>
>
>
> $ svn log -v -r 123385 http://HOST1/svnroot_nfs
>
> ==>>
> ------------------------------------------------------------------------
> r123385 | mvaradachari | 2008-09-25 17:39:39 -0700 (Thu, 25 Sep 2008) | 1 line
> Changed paths:
> D /tfb/branches/mvaradachari_b1/html/intern
>
> Deleting html/intern
>
> ==>>
>
>
>
> Q: Has anyone seen this type of behaviour before ?? The log of 123386
> indicates no change.
>
> The output of rev# 123386 from /svnroot_nfs/db/revs/123/123386
>
> Cat /svnroot_nfs/db/revs/123/123386
> ===>>>>
> id: 0.0.r123386/0
> type: dir
> pred: 0.0.r123385/29588
> count: 123386
> text: 123385 28688 887 887 1b234e9d9466d77921f61ebfaeb9bdd8
> cpath: /
> copyroot: 0 /
> is-fresh-txn-root: y
> minfo-cnt: 184
>
>
> 0 186
> ====>>
>
> The output of rev# 123385 from /svnroot_nfs/db/revs/123/123385
>
> cat /svnroot_nfs/db/revs/123/123385
> ===>>>
> ........
> ........
> ..........
> id: 0.0.r123385/29588
> type: dir
> pred: 0.0.r123384/0
> count: 123385
> text: 123385 28688 887 887 1b234e9d9466d77921f61ebfaeb9bdd8
> cpath: /
> copyroot: 0 /
> minfo-cnt: 184
>
> 8t-11.0-5930.r123372/55768 delete false false
> /tfb/branches/mvaradachari_b1/html/intern
>
>
> ===>>>
>
>
> Is this a bug ? Or did I miss something.
>
> Cheers
> murli
>
>
> On 9/23/08 7:59 AM, "david.x.grierson_at_jpmorgan.com
> <http://david.x.grierson@jpmorgan.com> " <david.x.grierson@jpmorgan.com
> <http://david.x.grierson@jpmorgan.com> > wrote:
>
>> Hi,
>>
>> We ran into problems with Subversion 1.4.5 using the worker MPM because
>> the apr-util library which controls the DAV activities database uses DBM
>> files to track changes.
>>
>> DBM files require POSIX locks to be able to be held on files - since
>> threads aren't real processes they don't obey POSIX locking properly and
>> so corruption to your repository can occur - particular when large commits
>> are taking place.
>>
>> Lieven Govaerts pointed out to me on the "Shared storage for subversion
>> repository" thread that "Subversion 1.5 has its own file-based mechanism
>> for maintaining the DAV activity mappings, so the above is no longer
>> true." So it could well be that the worker MPM may now be feasible.
>>
>> We're consuming 1TB of storage across 1056 repositories which are
>> receiving approx 3,500 commits per day. These are being hosted by a single
>> headed Apache host which has 2 x dual-core Xeon 2.66GHz and 16Gb of RAM.
>> The NFS data is held on a FAS3050 NetApp filer which is replicated to a DR
>> site via NetApp SnapMirror software. There is identical hardware in both
>> production and DR with identical software and configuration.
>>
>> Dg.
>> --
>> David Grierson
>> JPMorgan - IB Architecture - Source Code Management Consultant
>> GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson_at_jpmorgan.com
>> <http://david.x.grierson@jpmorgan.com>
>> Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
>>
>>
>>
>>
>> Murli Varadachari <mvaradachari_at_facebook.com
>> <http://mvaradachari@facebook.com> >
>> 23/09/2008 15:38
>>
>> To
>> Steve Whitson <steven.whitson@gmail.com <http://steven.whitson@gmail.com> >,
>> <users@subversion.tigris.org <http://users@subversion.tigris.org> >
>> cc
>> Internal Use <mvaradachari@facebook.com <http://mvaradachari@facebook.com> >
>> Subject
>> Re: subversion on NFS
>>
>>
>>
>>
>>
>>
>>
>> I have just set up a shared subversion repository [ on a NFS partition ]
>> and am in the process of testing simultaneous read-writes from a number of
>> hosts. SO far the tests seem ok. The current setup uses svn+ssh
>> protocols.
>>
>> Down the road I intend to use both http(s) and svn protocols.
>>
>> I am curious why you ran into corruption problems using "worker mpm" -- is
>> that still a problem with svn 1.5. [ on a fedora core linux ]
>>
>> Also you mentioned that you are using about 1TB of repository storage ‹ is
>> that a single repository.??
>>
>> Cheers
>> murli
>>
>>
>> On 9/23/08 7:19 AM, "Steve Whitson" <steven.whitson_at_gmail.com
>> <http://steven.whitson@gmail.com> > wrote:
>>
>> Thanks David, I appreciate your feedback!
>>
>> david.x.grierson@jpmorgan.com <http://david.x.grierson@jpmorgan.com> wrote:
>>> > Hi,
>>> >
>>> > We successfully run our services from NFS stored repositories using the
>>> > Apache prefork MPM serving over 1TB of repository storage transferring
>> an
>>> > average of over 60GB of traffic per day (1920.60Gb of data transferred
>> for
>>> > August).
>>> >
>>> > When I upgraded to 1.4.5 early this year I ran into repository
>> corruption
>>> > problems - these turned out to be because I had switched over to using
>> the
>>> > worker MPM.
>>> >
>>> > Regards,
>>> >
>>> > Dg.
>>> > --
>>> > David Grierson
>>> > JPMorgan - IB Architecture - Source Code Management Consultant
>>> > GDP 228-5574 / DDI +44 141 228 5574 / Email
>> david.x.grierson@jpmorgan.com <http://david.x.grierson@jpmorgan.com>
>>> > Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
>>> >
>>> >
>>> >
>>> >
>>> > Steve Whitson <steven.whitson@gmail.com <http://steven.whitson@gmail.com>
>
>>> > 22/09/2008 20:26
>>> >
>>> > To
>>> > "users@subversion.tigris.org <http://users@subversion.tigris.org> "
>>> <users@subversion.tigris.org <http://users@subversion.tigris.org> >
>>> > cc
>>> >
>>> > Subject
>>> > subversion on NFS
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > When serving subversion (1.5.2 fsfs) through apache (2.2.9)... (this is
>>> > on a Solaris 10 system with nfs from netapp).
>>> >
>>> > I see where subversion should not be used with the apache worker MPM
>>> > module. Is using the prefork MPM module ok?
>>> >
>>> > Another question I have, can I allow svnserve access to my set of
>>> > repositories (read/write) while at the same time serving them up through
>>> > apache (something I'm toying with) or is doing this going to eventually
>>> > cause corruption?
>>> >
>>> > I've read the FAQ on NFS, and several past threads:
>>> > http://subversion.tigris.org/faq.html#nfs
>>> > http://svn.haxx.se/users/archive-2008-05/0048.shtml
>>> > http://svn.haxx.se/dev/archive-2007-04/0288.shtml
>>> >
>>> > Thanks much,
>>> > -Steve
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
>>> <http://users-unsubscribe@subversion.tigris.org>
>>> > For additional commands, e-mail: users-help_at_subversion.tigris.org
>>> <http://users-help@subversion.tigris.org>
>>> >
>>> >
>>> >
>>> >
>>> > Generally, this communication is for informational purposes only
>>> > and it is not intended as an offer or solicitation for the purchase
>>> > or sale of any financial instrument or as an official confirmation
>>> > of any transaction. In the event you are receiving the offering
>>> > materials attached below related to your interest in hedge funds or
>>> > private equity, this communication may be intended as an offer or
>>> > solicitation for the purchase or sale of such fund(s). All market
>>> > prices, data and other information are not warranted as to
>>> > completeness or accuracy and are subject to change without notice.
>>> > Any comments or statements made herein do not necessarily reflect
>>> > those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>>> >
>>> > This transmission may contain information that is privileged,
>>> > confidential, legally privileged, and/or exempt from disclosure
>>> > under applicable law. If you are not the intended recipient, you
>>> > are hereby notified that any disclosure, copying, distribution, or
>>> > use of the information contained herein (including any reliance
>>> > thereon) is STRICTLY PROHIBITED. Although this transmission and any
>>> > attachments are believed to be free of any virus or other defect
>>> > that might affect any computer system into which it is received and
>>> > opened, it is the responsibility of the recipient to ensure that it
>>> > is virus free and no responsibility is accepted by JPMorgan Chase &
>>> > Co., its subsidiaries and affiliates, as applicable, for any loss
>>> > or damage arising in any way from its use. If you received this
>>> > transmission in error, please immediately contact the sender and
>>> > destroy the material in its entirety, whether in electronic or hard
>>> > copy format. Thank you.
>>> > Please refer to http://www.jpmorgan.com/pages/disclosures for
>>> > disclosures relating to UK legal entities.
>>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
>> <http://users-unsubscribe@subversion.tigris.org>
>> For additional commands, e-mail: users-help_at_subversion.tigris.org
>> <http://users-help@subversion.tigris.org>
>>
>>
>>
>>
>> Generally, this communication is for informational purposes only
>> and it is not intended as an offer or solicitation for the purchase
>> or sale of any financial instrument or as an official confirmation
>> of any transaction. In the event you are receiving the offering
>> materials attached below related to your interest in hedge funds or
>> private equity, this communication may be intended as an offer or
>> solicitation for the purchase or sale of such fund(s). All market
>> prices, data and other information are not warranted as to
>> completeness or accuracy and are subject to change without notice.
>> Any comments or statements made herein do not necessarily reflect
>> those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>>
>> This transmission may contain information that is privileged,
>> confidential, legally privileged, and/or exempt from disclosure
>> under applicable law. If you are not the intended recipient, you
>> are hereby notified that any disclosure, copying, distribution, or
>> use of the information contained herein (including any reliance
>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>> attachments are believed to be free of any virus or other defect
>> that might affect any computer system into which it is received and
>> opened, it is the responsibility of the recipient to ensure that it
>> is virus free and no responsibility is accepted by JPMorgan Chase &
>> Co., its subsidiaries and affiliates, as applicable, for any loss
>> or damage arising in any way from its use. If you received this
>> transmission in error, please immediately contact the sender and
>> destroy the material in its entirety, whether in electronic or hard
>> copy format. Thank you.
>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>> disclosures relating to UK legal entities.
>>
Received on 2008-09-29 16:50:44 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.