[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Multiple Subversion Repositories / Federated Repositories

From: Johnathan Gifford <jgifford_at_wernervas.com>
Date: 2006-12-15 16:16:29 CET

We've ran across the same need here in the USA with an office in China,
but for a different reason. Security and ownership of the code was the
issue. So our employees in China hit a second server here in the USA
and because of security (basically giving the China employees full
access of our source) they use a second very isolated server.

To solve the problem, we developed some shell scripts to sync the two
servers. We also determined that the US repository is where all the
merging to trunk would occur. So under branches, we created two
directories for each site (usa,china). Any issues, features, etc. would
be created in another directory under branches/location path.

To facilitate the synchronization, in the post-commit, we do an
incremental dump. We then over scp copy the file to the other server.
Then the post-commit uses ssh to execute a script on the other server
that loads the incremental dump on the other server.

Now this works great, but, if you run the hooks scripts again when you
load the dump file on the other server, you have to restrict commit
access for all users. Read is okay, but commit must be controlled. In
our case the China office can only commit to their branches/location
area. You have to do it this way, or you could end up getting a
redundant synchronization cycle going, which wouldn't be good. But you
don't have to run the hook scripts when loading the dump file unless you
want to. If you don't then just submit the dump.

I've been please with the results. It would be nice if there was an
out of the box solution or other options. The only drawback would be if
connectivity is an issue. There is no way send the file again. No if
you don't mind synchronizing on a scheduled basis, you could have the
server retrieve the dump files from the other server and then load them
one by one.

Johnathan

--------------------------------------------
Post-commit code
--------------------------------------------

# Copy commit to primary server and load into repository on remote
server
# if commit is from the China branch.
echo `date` copying and loading commit to primary server started
egrep "(Node-path)" /svn/backups/incremental/$DUMP_FILE_NAME | grep -c
--invert-match "branches/china" || {
        echo `date` copying to primary server
        /usr/bin/scp /svn/backups/incremental/$DUMP_FILE_NAME
account@svn.usa.domain.com:/svn/backups/sync/$REPOS_BASENAME
        echo `date` loading to primary server
        /usr/bin/ssh account@svn.usa.domain.com sh
/svn/scripts/load-commit $REPOS_BASENAME $DUMP_FILE_NAME
}
echo `date` copying and loading commit to primary server finished

--------------------------------------------
load-commit script
--------------------------------------------
#!/bin/bash

REPOS="$1"
FILE="$2"

SVN_HOME=/svn
SVN_BIN=/usr/local/bin

echo `date` >> $SVN_HOME/logs/load-commit.log
echo $REPOS >> $SVN_HOME/logs/load-commit.log
echo $FILE >> $SVN_HOME/logs/load-commit.log

$SVN_BIN/svnadmin load $SVN_HOME/repositories/$REPOS
--use-pre-commit-hook --use-post-commit-hook <
$SVN_HOME/backups/sync/$REPOS/$FILE

>>> On Fri, Dec 15, 2006 at 8:27 AM, in message
<359260C9CA8CC140A6F7E09A1F39D9E901401594@sssmail.toronto.sigmasoft.ca>,
"Steven Whatmore" <Stevenw@sigmasoft.ca> wrote:
> Good morning,
>
> I have a question regarding maintaining multiple synchronized
Subversion
> repositories or in other words a Federated Repository
>
> We have a distributed development team (one in Canada and another in
> India), where each team has the ability to update and commit code to
our
> code repository. Subversion includes a number of benefits for
minimizing
> the amount of data being sent in / out of the repository on commits,
> however due to the type of connection that our India team has (very
slow
> and unreliable) it makes sense to have a federated repository setup.
>
> When I say federated repository, I mean that each team (Canada /
India)
> both have a working master copy of the repository of essentially the
> same repository. Each team could treat their repository as a local
> repository and check code into / out of the repository, thus
mitigating
> any WAN network issues since the repositories are essentially local
to
> that team. However, in this case Subversion would do all of the work
in
> keeping those two repositories in sync. There would be of course
> situations where there would be conflicts on synchronization, but
those
> could be handled manually.
>
> synsync is not a solution for this problem in that synsync creates
> read- only mirrors and would not allow one of the teams to make
commits
> to their local repository.
>
> Any direction or assistance on this would be greatly appreciated.
>
> Steve Whatmore
> Technical Lead Web Development
> Sigma Software Solutions
> 55 York St., Suite 1100
> Toronto, ON
> M5J 1R7
>
> Phone: 416.368.2000 x2293
> Email: stevenw@sigmasoft.ca <mailto:stevenw@sigmasoft.ca>
> MSN: steven.whatmore@hotmail.com
<mailto:steven.whatmore@hotmail.com>
>
> Web: www.sigmasoft.ca <http://www.sigmasoft.canorthplains.com/>
> Skype: steven.whatmore <callto://steven.whatmore/>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Dec 15 16:17:50 2006

This is an archived mail posted to the Subversion Users mailing list.