[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Moving from one svn server to another, despite heavy use of absolute external references?

From: Hari Kodungallur <hkodungallur_at_gmail.com>
Date: Tue, 28 Jul 2009 16:05:08 -0700

On Tue, Jul 28, 2009 at 3:04 PM, Dan Stromberg <dstromberglists_at_gmail.com>wrote:

> On Tue, Jul 28, 2009 at 2:36 PM, Hari Kodungallur<hkodungallur_at_gmail.com>
> wrote:
> >
> > On Tue, Jul 28, 2009 at 1:56 PM, Dan Stromberg <
> dstromberglists_at_gmail.com>
> > wrote:
> >>
> >> We have 3 subversion repositories on a single subversion server, and
> >> we want to move them to a new subversion server.
> >>
> >> This seems simple enough - until you start to consider all the
> >> absolute external references (in branches and tags) we'd want to think
> >> about updating - and we're a little skittish about updating our tags.
> >>
> >> I calculated that for one of the 3 repos, if we have an automatic
> >> process that takes about a half hour to update one branch or tag, it'd
> >> take about 9 days of continuous updating to adjust all of our
> >> externals. That's kind of a long time, and I'm afraid the estimate is
> >> more likely low than high.
> >
> > Half hour to update one branch/tag? So, you have hundreds of sub
> directories
> > in each branch/tag and each directory has its own set of externals?
> >
> Including the checkout, update and checkin, yes, I think about a half
> hour is a pretty safe lower bound. Yes, we have hundreds of
> directories per branch/tag.
> >>
> >> There's also the option of generating a list of all the hostnames and
> >> IP's used in our external references and stealing those hostnames
> >> and/or IP's, so that the externals don't need to be updated.
> >
> > This is not a bad idea at all. Also, it likely is not a "list of"
> hostnames
> > and IPs; should be just one IP and hostname. Since the 3 repos were on
> one
> > single server, the externals should be using either the hostname of that
> > server or the IP of that server.
> We may do this, but I fear our IP's/hostnames aren't that consistent.
> I've found 3 different ones so far, and am no where near done scanning
> for them all.

May be I don't undertand this well enough. Assuming the externals are
referring to one of the 3 repos that reside on the same server, the
possibilities are: 'localhost', '', the hostname or the actual IP
address of the server. If there is another IP address referring to another
repository on another server, you don't need to change it because it is not
one of the 3 repos that you are moving.

> >
> >>
> >> Has someone else already been down this road?
> >>
> >> I have a tool that does a reasonable job of updating external
> >> references by piping them all through sed, but it's never been used on
> >> this scale. If there's something in common use, I'd probably use that
> >> instead. Is there something like this available?'
> >
> > Have you tried svnmucc? I believe there was also a script (python?) that
> > could be used to update the externals. I forget it now and can't seem to
> > find it.
> This is the first I've heard of svnmucc. Based on its usage()
> function, I'm guessing it'll set all of our externals to a fixed
> value, but not take an existing (and variable) svn:externals and
> change it according to a fixed pattern.

No it works on single URL. So, you still have to automate the process by
changing the external for each and every directory. You just save the time
required to checkout. Thats all. (Basically you don't need a working copy to
do this)
You will have to get each directory (svn dirs), get its externals (svn pget
url), update the hostnames (do some sed magic) and then use svnmucc to
update the external for that directory (svnmucc propset svn:external

> I don't see anything that was obviously an external updater at
> http://subversion.tigris.org/tools_contrib.html

Yep, I looked there, but could not find anything. I may be confusing it with
something else. I don't remember it either way :)

> > Also, what are the externals actually referring to? Are they directories
> > within the same repo? If that is the case, the latest svn supports
> relative
> > URLs. If you are taking the trouble to update the externals anyway, may
> be
> > you should make them relative URLs (wherever applicable) so that this is
> not
> > a problem in the future.
> Most of the externals are within the same repository. Some are to
> other repositories.
> Related topic:
> - I can't check out all of our source, without going through a ton of
> external references.
> - I want to check out as much as a I can from the new server, update
> the external references, and check back in.

Just to do this particular task, you don't need to checkout the externals
(using --ignore-externals in the checkout command)

> - But how can I be sure that I'm checking into the new server and not
> the old servrer, short of making the old server unreachable somehow?
> - Do I need to do a single svn switch for each branch and tag?
> - Do I need to do an svn switch for every directory having external
> references?
> >
> >>
> >>
> >> What are best practices on this sort of scenario?
> >>
> >>
> >> (No, I wouldn't have architected it this way at all, but I'm afraid
> >> we're stuck with it. I'd've used ant/make variables or environment
> >> variables instead, and failing that symlinks instead, and failing that
> >> relative external references, and failing that a DNS CNAME, or at
> >> absolute worst a series of _hosts_ files for the Subversion server)
> >>
> >> Thanks!
> >

I forgot about another method (that I have never used -- because I have
never needed to) that you could try. Assuming you have access to the
repository (server side), you can dump the repository and then I believe the
SvnDumpTool has an option (transform-prop) to fix up the externals. Please
try it out and see if it works for you.


To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2009-07-29 01:06:28 CEST

This is an archived mail posted to the Subversion Users mailing list.