[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

RE: subversion on NFS

From: Harvey, Edward <Edward.Harvey_at_patni.com>
Date: Mon, 29 Sep 2008 08:12:31 -0400

First of all, you shouldn't be accessing a svn repository via nfs mount. Try svnserve instead. It's easy, it's fast, and it guarantees you won't screw up your repository. Create a new user called "svn" or something like that, and only allow the svn user to access the repo. If you don't, then you will surely have some clever user accessing the repo directly, and screw it up anyway. If you need something more configurable than svnserve, apache's your friend, but apache requires a lot more work.

If for some reason you insist on using file:///<file:///\\> url to access the repo, and you're getting screwed up permissions (the least of your concerns) then do this... Make a group "mygroup" that includes everybody. chgrp -R mygroup /path/to/repo. And chmod -R g+ws /path/to/repo. And then you won't have permission problems anymore.

It's the group setgid bit.

From: Rafiuddin M K [mailto:rafiuddin.mk_at_gmail.com]
Sent: Monday, September 29, 2008 3:17 AM
To: Murli Varadachari
Cc: david.x.grierson_at_jpmorgan.com; Steve Whitson; users_at_subversion.tigris.org
Subject: Re: subversion on NFS

I am really new to SVN and Have a problem. Repeatedly, My Server where SVN is Installed Group is getting read only, though it having and showing WRITE/READ Permissions. Initially thought due to running of some Dump Scripts. But the problem resist after stopping that also.

can anyone suggest on this.....

Error: Commit failed (details follow):
Error: Can't create directory '/group/REPOSITORY/project/db/transactions/1578-1.txn': Read-only file system

--
Rafiuddin MK
On Mon, Sep 29, 2008 at 6:47 AM, Murli Varadachari <mvaradachari_at_facebook.com<mailto:mvaradachari_at_facebook.com>> wrote:
RE: Testing a shared subversion  repository [ netapps partition - NFS mounted ] with multiple access points [ HOST1, HOST2]. Test consisted of large parallel commits of the same set of changes .
Results: WEIRD ---  a "dummy" commit with no changes whatsoever  [ in one instance ]
I ran bunch of tests [ using a shared NFS storage ] primarily to determine if I could write / commit  simultaneously  from multiple hosts and whether the locking mechanisms work as advertised.
Configuration used
++++++++++++++++
(a) Shared svn repository located on a netapps partition - NFS mounted on two hosts HOST1, HOST2.
(b) HOST1 had a apache 2.2.x setup with per-fork mpm enabled and with full permissions to the svn repo.
User access via "http://URL"
(c) Access to HOST2  using svn+ssh protocol  also with full write access.
User access via "svn+ssh://URL"
(d) Locking enabled on netapps  + nfs hard mount [ default]
TEST
++++
1: I made a number of commits [ both small and big] using  svn+ssh and http protocols  from both HOST1 and HOST2 . Results were OK
2:  Then I made two large simultaneous commits after deleting a very large directory. Please note that I was deleting  the same set of files -- one based off HOST1 and the other one based off HOST2  [ http and svn+ssh protocols ]
Example:
svn delete  my-WC-HOST1/html/intern
svn delete my-WC-HOST2/html/intern
svn commit -m "Delete html/intern"  my-WC-HOST1  [  uses http://URL ]
svn commit -m "Delete html/intern"  my-WC-HOST2  [ uses svn+ssh://URL ]
RESULTS
++++++++
What I would have expected to see is the commit going through from one HOST and a error / failure / conflict from the 2nd HOST since the same set of files have already been deleted.
However the results were unexpected --- the commit from HOST1 went through OK - strangely the commit from HOST2 also went thorough. However the second commit log  indicated *NO* changes  -- a completely "dummy" commit generated via HOST2.
$ svn log -v -r 123386  http://HOST1/svnroot_nfs
==>>
------------------------------------------------------------------------
r123386 | mvaradachari | 2008-09-25 17:39:40 -0700 (Thu, 25 Sep 2008) | 1 line
Deleting html/intern
==>>
$ svn log -v -r 123385  http://HOST1/svnroot_nfs
==>>
------------------------------------------------------------------------
r123385 | mvaradachari | 2008-09-25 17:39:39 -0700 (Thu, 25 Sep 2008) | 1 line
Changed paths:
   D /tfb/branches/mvaradachari_b1/html/intern
Deleting html/intern
==>>
Q: Has anyone seen this type of behaviour before ?? The log of 123386 indicates no change.
The output of rev# 123386 from /svnroot_nfs/db/revs/123/123386
Cat  /svnroot_nfs/db/revs/123/123386
 ===>>>>
id: 0.0.r123386/0
type: dir
pred: 0.0.r123385/29588
count: 123386
text: 123385 28688 887 887 1b234e9d9466d77921f61ebfaeb9bdd8
cpath: /
copyroot: 0 /
is-fresh-txn-root: y
minfo-cnt: 184
0 186
====>>
The output of rev# 123385 from /svnroot_nfs/db/revs/123/123385
cat  /svnroot_nfs/db/revs/123/123385
===>>>
........
........
..........
id: 0.0.r123385/29588
type: dir
pred: 0.0.r123384/0
count: 123385
text: 123385 28688 887 887 1b234e9d9466d77921f61ebfaeb9bdd8
cpath: /
copyroot: 0 /
minfo-cnt: 184
8t-11.0-5930.r123372/55768 delete false false /tfb/branches/mvaradachari_b1/html/intern
===>>>
Is this a bug ? Or did I miss something.
Cheers
murli
On 9/23/08 7:59 AM, "david.x.grierson@jpmorgan.com<http://david.x.grierson@jpmorgan.com>" <david.x.grierson@jpmorgan.com<http://david.x.grierson@jpmorgan.com>> wrote:
Hi,
We ran into problems with Subversion 1.4.5 using the worker MPM because
the apr-util library which controls the DAV activities database uses DBM
files to track changes.
DBM files require POSIX locks to be able to be held on files - since
threads aren't real processes they don't obey POSIX locking properly and
so corruption to your repository can occur - particular when large commits
are taking place.
Lieven Govaerts pointed out to me on the "Shared storage for subversion
repository" thread that "Subversion 1.5 has its own file-based mechanism
for maintaining the DAV activity mappings, so the above is no longer
true." So it could well be that the worker MPM may now be feasible.
We're consuming 1TB of storage across 1056 repositories which are
receiving approx 3,500 commits per day. These are being hosted by a single
headed Apache host which has 2 x dual-core Xeon 2.66GHz and 16Gb of RAM.
The NFS data is held on a FAS3050 NetApp filer which is replicated to a DR
site via NetApp SnapMirror software. There is identical hardware in both
production and DR with identical software and configuration.
Dg.
--
David Grierson
JPMorgan - IB Architecture - Source Code Management Consultant
GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson@jpmorgan.com<http://david.x.grierson@jpmorgan.com>
Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
Murli Varadachari <mvaradachari@facebook.com<http://mvaradachari@facebook.com>>
23/09/2008 15:38
To
Steve Whitson <steven.whitson@gmail.com<http://steven.whitson@gmail.com>>, <users@subversion.tigris.org<http://users@subversion.tigris.org>>
cc
Internal Use <mvaradachari@facebook.com<http://mvaradachari@facebook.com>>
Subject
Re: subversion on NFS
I have just set up a shared subversion repository [ on a NFS partition ]
and am in the process of testing simultaneous read-writes from a number of
hosts.  SO far the tests seem ok. The current setup uses svn+ssh
protocols.
Down the road I intend to use both http(s) and svn protocols.
I am curious why you ran into corruption problems using "worker mpm" -- is
that still a problem with svn 1.5.  [ on a fedora core linux ]
Also you mentioned that you are using about 1TB of repository storage - is
that a single repository.??
Cheers
murli
On 9/23/08 7:19 AM, "Steve Whitson" <steven.whitson@gmail.com<http://steven.whitson@gmail.com>> wrote:
Thanks David,  I appreciate your feedback!
david.x.grierson@jpmorgan.com<http://david.x.grierson@jpmorgan.com> wrote:
> Hi,
>
> We successfully run our services from NFS stored repositories using the
> Apache prefork MPM serving over 1TB of repository storage transferring
an
> average of over 60GB of traffic per day (1920.60Gb of data transferred
for
> August).
>
> When I upgraded to 1.4.5 early this year I ran into repository
corruption
> problems - these turned out to be because I had switched over to using
the
> worker MPM.
>
> Regards,
>
> Dg.
> --
> David Grierson
> JPMorgan - IB Architecture - Source Code Management Consultant
> GDP 228-5574 / DDI +44 141 228 5574 / Email
david.x.grierson@jpmorgan.com<http://david.x.grierson@jpmorgan.com>
> Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
>
>
>
>
> Steve Whitson <steven.whitson@gmail.com<http://steven.whitson@gmail.com>>
> 22/09/2008 20:26
>
> To
> "users@subversion.tigris.org<http://users@subversion.tigris.org>" <users@subversion.tigris.org<http://users@subversion.tigris.org>>
> cc
>
> Subject
> subversion on NFS
>
>
>
>
>
>
> When serving subversion (1.5.2 fsfs) through apache (2.2.9)... (this is
> on a Solaris 10 system with nfs from netapp).
>
> I see where subversion should not be used with the apache worker MPM
> module.  Is using the prefork MPM module ok?
>
> Another question I have, can I allow svnserve access to my set of
> repositories (read/write) while at the same time serving them up through
> apache (something I'm toying with) or is doing this going to eventually
> cause corruption?
>
> I've read the FAQ on NFS, and several past threads:
>     http://subversion.tigris.org/faq.html#nfs
>     http://svn.haxx.se/users/archive-2008-05/0048.shtml
>     http://svn.haxx.se/dev/archive-2007-04/0288.shtml
>
> Thanks much,
>     -Steve
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org<http://users-unsubscribe@subversion.tigris.org>
> For additional commands, e-mail: users-help@subversion.tigris.org<http://users-help@subversion.tigris.org>
>
>
>
>
> Generally, this communication is for informational purposes only
> and it is not intended as an offer or solicitation for the purchase
> or sale of any financial instrument or as an official confirmation
> of any transaction. In the event you are receiving the offering
> materials attached below related to your interest in hedge funds or
> private equity, this communication may be intended as an offer or
> solicitation for the purchase or sale of such fund(s).  All market
> prices, data and other information are not warranted as to
> completeness or accuracy and are subject to change without notice.
> Any comments or statements made herein do not necessarily reflect
> those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to UK legal entities.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org<http://users-unsubscribe@subversion.tigris.org>
For additional commands, e-mail: users-help@subversion.tigris.org<http://users-help@subversion.tigris.org>
Generally, this communication is for informational purposes only
and it is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation
of any transaction. In the event you are receiving the offering
materials attached below related to your interest in hedge funds or
private equity, this communication may be intended as an offer or
solicitation for the purchase or sale of such fund(s).  All market
prices, data and other information are not warranted as to
completeness or accuracy and are subject to change without notice.
Any comments or statements made herein do not necessarily reflect
those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to UK legal entities.
________________________________
This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, e-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error kindly delete this e-mail from your records. If it appears that this mail has been forwarded to you without proper authority, please notify us immediately at netadmin_at_patni.com and delete this mail.
Received on 2008-09-29 14:13:13 CEST

This is an archived mail posted to the Subversion Users mailing list.