[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: SVN Externals 1.6 to 1.7 migration issue

From: Johan Corveleyn <jcorvel_at_gmail.com>
Date: Thu, 2 May 2013 15:18:00 +0200

On Thu, May 2, 2013 at 12:50 PM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
> On Thu, May 2, 2013 at 11:00 AM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
>> On Thu, May 2, 2013 at 9:55 AM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
>>> On Wed, May 1, 2013 at 5:01 PM, BRM <bm_witness_at_yahoo.com> wrote:
>>>> While I have not had the issue you are having, assuming the externals are in
>>>> the same repository I would highly recommend changing from using the syntax
>>>> you have to using the carrot (^) operator as it will save you many headaches
>>>> if your original svn location changes.
>>>> I.e. use:
>>> Indeed, that's far better than using an absolute url with scheme etc.
>>>> Also, I think the specific issue you are having (looking more closely at the
>>>> error message below) is that AFAIK, SVN does not let you import specific
>>>> files via externals - you have to do name spaces (aka folders) instead.
>>> That's not correct. File externals are supported (since 1.6 I think).
>>> They have a couple of specific problems though (a lot of which have
>>> been fixed in 1.7), because their implementation is entirely different
>>> from directory externals (file externals use the "switch" mechanism
>>> underneath, while dir externals are essentially an embedded checkout
>>> with some sugar on top).
>>> So, concerning to the problem of the OP:
>>>> From: "Hutchinson, Steve (UK)" <steven.hutchinson_at_mbda-systems.com>
>>>> To: "users_at_subversion.apache.org" <users_at_subversion.apache.org>
>>>> Sent: Wednesday, May 1, 2013 7:01 AM
>>>> Subject: SVN Externals 1.6 to 1.7 migration issue
>>>> Hi,
>>>> We're been using 1.6 svn externals to manage a FW task.
>>>> In the repository we have a folder structure (which is pretty much defined
>>>> by the tools we are using) as below :-
>>>> FOLDER A -> file_a1.txt, file_a2.txt, file_a3.txt
>>>> FOLDER B -> file_b1.txt, file_b2.txt, file_b3.txt
>>>> Then using externals we link to those folder to create a WC PROJECT
>>>> structure that looks like :-
>>>> PROJECT -> FOLDER C -> file_a1.txt, file_a2.txt, file_a3.txt AND
>>>> file_b1.txt
>>>> PROJECT -> FOLDER D -> file_b1.txt, file_b2.txt, file_b3.txt
>>>> The externals on PROJECT FOLDER we used looked like (not actually using file
>>>> protocol, just created for example purposes) :-
>>>> file:///D:/FPGA/SVN_ISSUE/repo/FOLDER_B/file_b1.txt FOLDER_C/file_b1.txt
>>>> Worked ok with 1.6. We get the below error in 1.7 when updating of :-
>>>> External failed: D:\FPGA\SVN_ISSUE\PROJECT\FOLDER_C\file_b1.txt
>>>> Error: Cannot insert a file external defined on
>>>> 'D:\FPGA\SVN_ISSUE\PROJECT' into the
>>>> Error: working copy 'D:\FPGA\SVN_ISSUE\PROJECT\FOLDER_C'.
>>>> We have tried a few things (will not share all at this point due to info
>>>> overload), but wondered if there were any others that achieve something like
>>>> this with 1.7 or could offer some advise ?
>>> So essentially you are "injecting" a file external (file_b1.txt)
>>> inside a directory external (FOLDER_C (an external of FOLDER_A)).
>>> Hm, I remember some past discussions about edge cases like these
>>> (during the working copy rewrite of 1.7), but I don't remember exactly
>>> if this is actually a supported use case, or if it just happened to
>>> work by accident in 1.6.
>>> Perhaps someone else on this list can shed some light on this?
>> I had the following conversation with Philip Martin, on IRC (#svn-dev):
>> <@jcorvel> philipm, perhaps you know the answer to this externals
>> question (works in 1.6, but no longer in 1.7):
>> http://svn.haxx.se/users/archive-2013-05/0006.shtml
>> <philipm> A file external defined in one working copy but located
>> inside another working copy is not going to work in 1.7 or 1.8.
>> <philipm> The directory external is a separate working copy with its
>> own .svn/wc.db and that knows nothing about the external defined in a
>> different .svn/wc.db
>> <@jcorvel> Ok. Sooo, is this more in the "it worked by accident in
>> 1.6, but was never supposed to be supported" category? Or could this
>> be considered a regression?
>> <@jcorvel> I guess it doesn't matter ... semantics ...
>> <philipm> I don't suppose we ever supported it deliberately.
>> <@jcorvel> perhaps the more interesting question then is: is this at
>> all supportable within the externals design (of wc-ng)? Or would this
>> require "externals 2.0"?
>> <philipm> If we change directory externals to be more like switch then
>> the directory externals become part of a single working copy.
>> <@jcorvel> ack
>> <philipm> It may be relatively simple to do that. At present if you
>> try to "svn sw URL path" and path does not exist then the switch
>> fails.
>> <philipm> But it only fails because of a high level check on path, in
>> the past it was possible to switch a non-existant path.
>> <philipm> (In the past being during 1.7 development)
>> <philipm> If you switch a path that does not exist that looks very
>> much like an external (it is how file externals are implemented)
>> <philipm> One of the difficulties about externals is that the
>> specification is so relaxed that there are so many different cases.
>> <philipm> Almost any change is going to break some untested corner
>> cases, while perhaps fixing other corner cases.
>> <@jcorvel> philipm: so, realistically, this will probably not be fixed
>> very soon (as in: in a 1.8.x release). At the very least, it needs (a
>> lot of) work for coming up with a detailed spec etc ... to try and get
>> a grip on the edge cases etc ...
>> <philipm> I don't think it will be fixed in the near future. I don't
>> know what we are going to do about externals.
>> So I guess that answers it. It's not going to work in the near future.
>> Short term I guess you have to find a workaround by structuring things
>> differently. Longer term, someone needs to "champion" an effort to
>> rework externals (starting by driving a discussion about it's
>> specification etc ...). Volunteers welcome :-).
> And some additional context was given by Bert Huijben:
> <@Bert> jcorvel: Injecting file externals into a separate working copy
> was deemed to be a never intended feature and a potential security
> problem. It was explicitly disabled during 1.7 development.
> <@Bert> jcorvel: The proper fix is part of the externals redesign to
> something like viewspecs... where the file external just becomes a
> working copy containing one file, but properly defined by some
> ancestor
> <@Bert> A problem with this 'feature' was that it was possible to
> inject such externals, but not to remove them. The working copy that
> contained the external didn't even contain the definition.

FWIW, "injecting" externals inside other externals seems to "work" in
1.7 when you do it with "directory externals" (for some definition of
"work" ... I haven't tried all use cases with it). However, I wouldn't
recommend it (it feels pretty wonky to me -- not sure if there aren't
any side effects -- for instance, 'svn status' output looks a bit
strange, see below).

Here is a transcript of a quick experiment with SVN 1.7.7:

C:\Temp\svntest2>svnadmin create repos

C:\Temp\svntest2>svn co file:///C:/Temp/svntest2/repos wc
Checked out revision 0.

C:\Temp\svntest2>cd wc

C:\Temp\svntest2\wc>svn mkdir dir1 dir2
A dir1
A dir2

C:\Temp\svntest2\wc>svn ci -mm
Adding dir1
Adding dir2

Committed revision 1.

C:\Temp\svntest2\wc>svn up
Updating '.':
At revision 1.

C:\Temp\svntest2\wc>svn pe svn:externals . # edit externals in text editor
Set new value for property 'svn:externals' on '.'

C:\Temp\svntest2\wc>svn pg svn:externals
^/dir1 ext_dir1
^/dir2 ext_dir1/ext_dir2

C:\Temp\svntest2\wc>svn ci -mm
Sending .

Committed revision 2.

C:\Temp\svntest2\wc>svn up
Updating '.':

Fetching external item into 'ext_dir1':
External at revision 2.

Fetching external item into 'ext_dir1\ext_dir2':
External at revision 2.

At revision 2.

C:\Temp\svntest2\wc>svn st
X ext_dir1

Performing status on external item at 'ext_dir1':
? C:\Temp\svntest2\wc\ext_dir1\ext_dir2

Performing status on external item at 'ext_dir1\ext_dir2':

C:\Temp\svntest2\wc>echo test > dir1\file.txt

C:\Temp\svntest2\wc>svn add dir1\file.txt
A dir1\file.txt

C:\Temp\svntest2\wc>svn ci -mm
Adding dir1\file.txt
Transmitting file data .
Committed revision 3.

C:\Temp\svntest2\wc>svn up
Updating '.':

Fetching external item into 'ext_dir1':
A ext_dir1\file.txt
Updated external to revision 3.

Fetching external item into 'ext_dir1\ext_dir2':
External at revision 3.

At revision 3.

C:\Temp\svntest2\wc>echo blah > dir2\file.txt

C:\Temp\svntest2\wc>svn add dir2\file.txt
A dir2\file.txt

C:\Temp\svntest2\wc>svn ci -mm
Adding dir2\file.txt
Transmitting file data .
Committed revision 4.

C:\Temp\svntest2\wc>svn up
Updating '.':

Fetching external item into 'ext_dir1':
External at revision 4.

Fetching external item into 'ext_dir1\ext_dir2':
A ext_dir1\ext_dir2\file.txt
Updated external to revision 4.

At revision 4.

C:\Temp\svntest2\wc>svn st
X ext_dir1

Performing status on external item at 'ext_dir1':
? C:\Temp\svntest2\wc\ext_dir1\ext_dir2

Performing status on external item at 'ext_dir1\ext_dir2':

C:\Temp\svntest2\wc>cat ext_dir1\file.txt

C:\Temp\svntest2\wc>cat ext_dir1\ext_dir2\file.txt

Received on 2013-05-02 15:18:52 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.