[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Creating and using child repositories of repositories which contain externals.

From: Looking through a Glass Onion. <zentechno_at_yahoo.com>
Date: Tue, 7 Oct 2008 11:39:50 -0700 (PDT)

Hi, again, SVN folks!

First off, my apologies for not adhering to your request for limiting
email to 72-columns! I though my mailer auto-formatted my email, but it
apparently doesn't. I've included my original post in 72-column
friendly format, so if you ignored my email yesterday for this reason I
understand, though I'd like you to reconsider. 8^) I also have a few
things to add which I've considered/discovered since posting yesterday.

First a quick review relevant to what I'd like to add today:

    MR -- Main Repository, shared by all engineers.
    LR -- Local Repository, used by a single engineer.
    LW -- Local Workspace, a checkout of MR, imported to LR.
    WS -- a "generic" workspace, though strictly a checkout of LR.

There's also a directory layout included in my original email below
detailing the layout of the master and child repository, and externals
to which I refer (at least well enough to have this discussion).

The basic idea was to create a Local Repository, and populate it with
the checkout of the Main Repository, then use that as a synchronization
place to co/ci files that are externals so the deltas can be propagated
to all places within the Local Repository so the changes can be tested
prior to committing them to the Master Repository.

OK -- to add to this, I discovered the "--ignore-externals" option to
`svn export` -- though this helps with the final checkin to the
Main Repository, it doesn't help at the 'lower' layer where I need my
child repository to understand the externals so the deltas of a checked
in file can be replicated to its (origin and) externals -- the whole
point here as this facilitates 'test-before-commit' in all places that
externals is populated in the checkout. If the `svn import` option
accepted the same "--ignore-externals" option, I'd be golden here! Is
there an effort underway to add this, and/or to those who understand
this more deeply than I would this be a useful addition?

I'm writing a script to delete the on-disk copies of the externals in a
workspace (by that I mean the checkout of a repository). In this way
just the non-externals/locals can be imported into he child repository
(amounting to an `svn import --ignore-externals` ...), then another
script to re-create the externals in the child repository which simply
uses `svn propget -R svn:externals trunk` to build the list from the
Local Workspace, then re-creates them in the child repository.

I'd be happy to contribute this script and whatever I end up doing to
make this work -- provided I get it working -- if it seems like it could
useful to the effort here.

As a final note, I haven't overlooked the possibility that I'm missing
something very easy here! Please advise if you think I'm well off the

Thanks for all your time.


--- On Mon, 10/6/08, Looking Through A Glass Onion <zentechno_at_yahoo.com>

> Begin forwarded message (cut-and-pasted from the web archive)
Date: Mon, 6 Oct 2008 16:32:10 -0700 (PDT)
From: Looking through a Glass Onion. <zentechno_at_yahoo.com>
Content-Type: text/plain; charset=us-ascii
Subject: Creating and using child repositories of repositories which
    contain externals.

 Hi SVN people!

 I've checked through the archives of this alias, and I've seen a few
 queries regarding setting up local (a.k.a. 'child') repositories of
 other repositories so version control can be done w/o network
 connectivity, then committed 'en masse' to the main repository at
 convenient times. The recommendation for this was to use SVK at
 http://svk.bestpractical.com/ -- but they don't support externals
 (at least not as of today's visit to
 but I am investigating the 'proposal' and 'workaround'). I don't
 think vendor branches work here either. In the mean time...

 My company's svn repository uses svn:externals in probably a unique
 way. We wanted to use a view-like mechanism, so we have a 'trunk' set
 up that contains all our components, and also contains a 'views'
 folder. The various views in the views folder contain sub-folders
 which are externals to the components of that view. For example:

    /trunk/componentX #some component
    /trunk/componentY #some other component
    /trunk/componentZ #yet another component
    /trunk/componentT #one last component

    /trunk/views/A/componentX #an external to /trunk/componentX
    /trunk/views/A/componentY #an external to /trunk/componentY
    /trunk/views/A/componentZ #an external to /trunk/componentZ

    /trunk/views/B/componentZ #an external to /trunk/componentZ
    /trunk/views/B/componentT #an external to /trunk/componentT

    In this way changes to the components code is automagically
 propagated into the 'views' -- however (you may know where I'm going)
 -- if I make a change to /trunk/views/A/componentZ it doesn't show in
 /trunk/component/views/B/comonentZ until I commit the change. This is
 somewhat anathema to a 'test-before-commit' methodology, as no matter
 how much I test componentZ in the A view, there's no way to test-
 before-commit in the B view unless I cold-copy the changes into
 trunk/views/B, which creates other oddities for subversion when it
 comes time to commit (or if not, because I limit the scope of my
 commit, then during the update of those folders).

    A solution I'd like to roll out in my own workspace(s) is to create
 a child/local repository of code from our main repository. Then, by
 checking out and committing via this child-repository I can synchronize
 my changes in one place, test them in the effected views, then commit
 these changes from my child/local repository back to the main
 repository. This would work pretty seamlessly, actually, if it wasn't
 for these externals (e.g. assuming for a moment no externals):

    -- bring over the main repository code
         `svn checkout http://svn/mainRepository/trunk ./MainTrunk`

    -- create the child/local repository
         `svnadmin create ./childRepository`

    -- import the main code into the child repository
         `svn import ./MainTrunk ./childRepository/trunk`

    -- run svnserve
         `svnserve -d -r ./childRepository

    At this point I can co, update, ci any changes via the child
 repository (e.g. on another machine, or to a different folder on the
 same machine):

         `svn co svn://<svnserveMachine>/trunk ./trunk`

    When it comes time to check these back into the MainTrunk, I simply:

    -- export childRepository code into the directory where the
           mainRepository is located (using loopback to make the point):

         `svn --export --force svn:// ./childRepository`

    This 'materializes' the changes made to the child repository into
 the code that was originally checked out of the main repository.

    At this point, I simply ci this code:

         `svn ci ./childRepository

    and the code goes into the mainRepository.

    That said, externals make this much more involved. During the
 import phase, svn is smart enough to ignore the '.svn' folders, BUT
 it's not smart enough to ignore the externals. This mean that in the
 child repository, the externals are treated like separate files, and so
 changes to them (e.g. via another checkout workspace) do NOT get
 propagated at commit time. E.g. using the layout above, a change to


 does NOT get propogated to


 because the child repository sees them as different files.

    I can see two ways to make this happen with externals (and to
 finally get to my questions, please tell me if this seems like this/
 either should work):

    Option 1:

    1) after the checkout of mainRepository, but before importing the
 source files into the child repository, remove all the externals.
 After the import, these folders can be easily replaced with another
 checkout of the mainRepository.

    2) after the import to the child repository, re-create the externals
 in the childRepository by simply replicating the `svn propset
 svn:externals ...` commands (e.g. create a shell script that uses
 `svn propget svn:externals ...` of the main repository code to set the
 externals of the child repository code).

    In this way, a change to a checkout of the child repository will get
 propagated to the externals folders with an update -- BUT what will
 happen during the export of the childRepository into the source of the
 checkout of the mainRepository?

        -- If these changes are propagated, as I think it (of course)
 will, then we have the same problem as using a single checkout and
 copying the changes into all 'views' -- namely multiple changes to the
 same file (as svn is designed to complain about).

        -- If these changes are NOT propagated then there seems to be no
 issues, and a subsequent checkin of this code into the mainRepository
 should go correctly.

    Option 2 (again, to the meat of a meaninful question): Is there
 some way to get svn to ignore these externals during the import as it
 does the '.svn' folders and even better, to also ignore these during
 the export, too!)? If so, this seems like a much better solution as
 it's all internal to svn, and requires no special handling time or
 risks to folders and code.

    Option 3 -- open to ideas!

    Thanks for your time reading through all this. Looking forward to
 some solution even if it involves shell scripts and/or changes to
 repository hooks!


  "All programs can be reduced to one line of code with a bug in it"


To unsubscribe, e-mail: users-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: users-help_at_subversion.tigris.org
Received on 2008-10-07 20:40:24 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.