Hi Nico, Uli and All,
Thanks for your replies.
You got me right that I am willing to implement a write-through-proxy set up where every WRITE request is redirected to master (server) by slave (server) while every READ request is served locally by the slave. I understand the risk of multiple users committing at the same time and currently I am not worried about it as the purpose of this setup is limited to very few users and I will look to have a work around of this problem later on, once this setup gets going.
So I changed my setup to work with HTTP rather on HTTP as I am having issue in accepting ssl-certificate from master. For the time being I will work with HTTP once it starts working I will move it to HTTPS. I am again describing the steps and configuration so that no confusion remains.
ON MASTER (server)
/etc/apache2/mods-available/dav_svn.conf
<Location /svn>
DAV svn
SVNParentPath /home/svn
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
</Location>
post-commit
/usr/bin/svnsync sync --username mirrorsync --password 123 --non-interactive http://slave-ip/svn-proxy-sync/mirror-repo
post-revprop-change
/usr/bin/svnsync copy-revprops --username mirrorsync --password 123 http://slave-ip/svn-proxy-sync/mirror-repo $REV
ON SLAVE (server)
/etc/apache2/mods-available/dav_svn.conf
<Location /svn>
DAV svn
SVNParentPath /home/svn
SVNMasterURI http://master-ip/svn
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
</Location>
<Location /svn-proxy-sync>
DAV svn
SVNParentPath /home/svn
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
</Location>
start-commit
if [ "$USER" = "mirrorsync" ];
then exit 0;
fi
echo "Only the mirrorsync user may commit new revisions" >&2 exit 1
pre-revprop-change
if [ "$USER" = "mirrorsync" ];
then exit 0;
fi
echo "Only the mirrorsync user may change revision properties" >&2 exit 1
"mirrorsync" is the user that exists in the /etc/apache2/dav_svn.passwd file.
In the above set up if as soon as something is COMMITED to master (directly) without involving slave it gets synchronized with the slave as well.
If I try to checkout from the slave it also works fine.
BUT if I try to commit to the slave I get error
"svn: Commit failed (details follow):"
"svn: Could not open the requested SVN filesystem"
This error is present inside the error.log file of the master so I assume the COMMIT request was redirected to master but it couldn't open the svn repository on the master.
The group and the user to which svn repository belong is "www-data" on both master and slave with rwx permissions.
Svn repository resides at /home/svn/master-repo on MASTER and at /home/svn/mirror-repo on SLAVE
I hope this information will help you to find some solution to my problem.
Thanks
Gaurav
-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel_at_gmail.com]
Sent: Thursday, August 11, 2011 12:35 AM
To: users_at_subversion.apache.org
Subject: Re: Help Needed regarding svn master-slave configuration
On Wed, Aug 10, 2011 at 8:04 AM, Ulrich Eckhardt
<ulrich.eckhardt_at_dominolaser.com> wrote:
> On Wednesday 10 August 2011, Nico Kadel-Garcia wrote:
>> On Wed, Aug 10, 2011 at 3:40 AM, Ulrich Eckhardt
>> <ulrich.eckhardt_at_dominolaser.com> wrote:
> [..]
>> > 3. connect the repositories
>> > This involves creating a post-commit hook that svnsyncs the changes to
>> > the secondary repository and one for the rev-prop changes, but you seem
>> > to have these set up already.
>> > 4. create a transparent write-proxy
>> > This requires Apache (the rest could be dove with svnserve, too) and
>> > should also be documented. I never made this step though but only the
>> > former three, since the write-proxy wasn't my requirement.
>>
>> Step 3 and 4 *are not enough*. The synchronization would have to occur
>> at the "commit" stage, not the "post-commit", because "commit" has to
>> be atomic with Subversion. Post-commit is nominally allowed to fail
>> without stopping the commit operation. And the only support I've seen
>> for that sort of thing is in Wandisco's "Multi-Site" tool.
>
> I don't think I understand you. Or maybe you don't understand me....
>
> Step three above means that new revisions in the primary repository are synced
> to the secondary repository. I didn't explicitly say that this is a one-way
> operation, but that's what I meant. In no case can you directly commit to the
> secondary repository.
The implication, from the original poster's question and, I thought,
from your response, was that you could do this on the *slave* server,
accept commits there, and push the changes to the master. This is what
I'm raising red flags: resolving potential "split-bran" operations due
to a master server being offline has to be done at the commit stage,
not the post-commit stage.
Since this is not what you meant, I am re-assured.
> Step four is a setup of Apache that simply redirects any write request to the
> primary repository while satisfying read requests from the secondary
> repository mirror.
That's not so dangerous, but potentially quite confusing if the slave
server is out of sync. Picture, if you will:
* Master server has a 4 Gig binary file, a DVD iso image, in
repo/branches/test-repo-1/dist.iso
* Developer 1 commits modified dist.iso on the master server,
revision 1001..
* A post-commit tries to publish the revision 1001 to the slave
server (svnsync operation)
* This takes time due to slow connection, say 10 minutes.
* During that 10 minutes, the slave server cannot report
knowledge about the changes in dist.iso: it's still at revision 1000.
* Developer 2 tries to commit modified dist.iso in the same branch.
* Commit operation reports missing update with passthrough
operations to master server.
* Developer 2 runs subversion update from "slave" server, which
is permitted, gets revision 1000 until and unless svnsync is complete.
* Developer 2 tries to commit modified dist.iso again,
* Developer 2 remains unable to commit change until svnsync is
complete, due to mismatch between revisions for read operations, and
write operations, because they are *2 distinct and out of sync
Subversion servers*, despite best efforts.
At this point, with luck, someone can explain to developer 2 that
there's a slight delay with bulky files that will be resolved. But
while that bulky revision is being pushed, *ALL* other revisions are
blocked, and *any active files on the master repository* can have
similar commit issus.
> I don't see this happening in any proper setup. The setup is not a net of peer
> repositories but one primary repository and multiple secondaries. Changes will
> only be made to the primary and then synced to the secondary.
That was unclear: thank you for attempting to clarify it. Gaurav
wanted to be able to commit changes from the secondary, to the master.
You've avoided that, in theory, if you can get the pass-through proxy
working, although I think I've described some potential issues for it.
> What can happen (and, at least for a short latency, always does) is that the
> secondary repository falls behind the primary, so someone updating from that
> repository sees an older state than someone using the primary. This doesn't
> cause severe problems though, at least not any more than those you get from a
> broken connection to the primary which causes all commits to fail.
One hopes. The "it worked yesterday, why doesn't it work today!!!???"
cry of upset developers is the sort of thing that can get engineers
who manage source control fired, unless they can get very clear what
they can and cannot reliably support, and get it in writing.
The information contained in this electronic mail transmission
may be privileged and confidential, and therefore, protected
from disclosure. If you have received this communication in
error, please notify us immediately by replying to this
message and deleting it from your computer without copying
or disclosing it.
Received on 2011-08-11 09:40:47 CEST