[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Multiple Subversion Repositories / Federated Repositories

From: Johnathan Gifford <jgifford_at_wernervas.com>
Date: 2006-12-15 16:16:29 CET

We've ran across the same need here in the USA with an office in China,
but for a different reason. Security and ownership of the code was the
issue. So our employees in China hit a second server here in the USA
and because of security (basically giving the China employees full
access of our source) they use a second very isolated server.

To solve the problem, we developed some shell scripts to sync the two
servers. We also determined that the US repository is where all the
merging to trunk would occur. So under branches, we created two
directories for each site (usa,china). Any issues, features, etc. would
be created in another directory under branches/location path.

To facilitate the synchronization, in the post-commit, we do an
incremental dump. We then over scp copy the file to the other server.
Then the post-commit uses ssh to execute a script on the other server
that loads the incremental dump on the other server.

Now this works great, but, if you run the hooks scripts again when you
load the dump file on the other server, you have to restrict commit
access for all users. Read is okay, but commit must be controlled. In
our case the China office can only commit to their branches/location
area. You have to do it this way, or you could end up getting a
redundant synchronization cycle going, which wouldn't be good. But you
don't have to run the hook scripts when loading the dump file unless you
want to. If you don't then just submit the dump.

I've been please with the results. It would be nice if there was an
out of the box solution or other options. The only drawback would be if
connectivity is an issue. There is no way send the file again. No if
you don't mind synchronizing on a scheduled basis, you could have the
server retrieve the dump files from the other server and then load them
one by one.


Post-commit code

# Copy commit to primary server and load into repository on remote
# if commit is from the China branch.
echo `date` copying and loading commit to primary server started
egrep "(Node-path)" /svn/backups/incremental/$DUMP_FILE_NAME | grep -c
--invert-match "branches/china" || {
        echo `date` copying to primary server
        /usr/bin/scp /svn/backups/incremental/$DUMP_FILE_NAME
        echo `date` loading to primary server
        /usr/bin/ssh account@svn.usa.domain.com sh
/svn/scripts/load-commit $REPOS_BASENAME $DUMP_FILE_NAME
echo `date` copying and loading commit to primary server finished

load-commit script



echo `date` >> $SVN_HOME/logs/load-commit.log
echo $REPOS >> $SVN_HOME/logs/load-commit.log
echo $FILE >> $SVN_HOME/logs/load-commit.log

$SVN_BIN/svnadmin load $SVN_HOME/repositories/$REPOS
--use-pre-commit-hook --use-post-commit-hook <

>>> On Fri, Dec 15, 2006 at 8:27 AM, in message
"Steven Whatmore" <Stevenw@sigmasoft.ca> wrote:
> Good morning,
> I have a question regarding maintaining multiple synchronized
> repositories or in other words a Federated Repository
> We have a distributed development team (one in Canada and another in
> India), where each team has the ability to update and commit code to
> code repository. Subversion includes a number of benefits for
> the amount of data being sent in / out of the repository on commits,
> however due to the type of connection that our India team has (very
> and unreliable) it makes sense to have a federated repository setup.
> When I say federated repository, I mean that each team (Canada /
> both have a working master copy of the repository of essentially the
> same repository. Each team could treat their repository as a local
> repository and check code into / out of the repository, thus
> any WAN network issues since the repositories are essentially local
> that team. However, in this case Subversion would do all of the work
> keeping those two repositories in sync. There would be of course
> situations where there would be conflicts on synchronization, but
> could be handled manually.
> synsync is not a solution for this problem in that synsync creates
> read- only mirrors and would not allow one of the teams to make
> to their local repository.
> Any direction or assistance on this would be greatly appreciated.
> Steve Whatmore
> Technical Lead Web Development
> Sigma Software Solutions
> 55 York St., Suite 1100
> Toronto, ON
> M5J 1R7
> Phone: 416.368.2000 x2293
> Email: stevenw@sigmasoft.ca <mailto:stevenw@sigmasoft.ca>
> MSN: steven.whatmore@hotmail.com
> Web: www.sigmasoft.ca <http://www.sigmasoft.canorthplains.com/>
> Skype: steven.whatmore <callto://steven.whatmore/>

To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Dec 15 16:17:50 2006

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.