Re: SVN Externals 1.6 to 1.7 migration issue

On Thu, May 2, 2013 at 11:00 AM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
> On Thu, May 2, 2013 at 9:55 AM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
>> On Wed, May 1, 2013 at 5:01 PM, BRM <bm_witness_at_yahoo.com> wrote:
>>> While I have not had the issue you are having, assuming the externals are in
>>> the same repository I would highly recommend changing from using the syntax
>>> you have to using the carrot (^) operator as it will save you many headaches
>>> if your original svn location changes.
>>>
>>> I.e. use:
>>>
>>> ^/FOLDER_A FOLDER_C
>>> ^/FOLDER_B FOLDER_D
>>
>> Indeed, that's far better than using an absolute url with scheme etc.
>>
>>> Also, I think the specific issue you are having (looking more closely at the
>>> error message below) is that AFAIK, SVN does not let you import specific
>>> files via externals - you have to do name spaces (aka folders) instead.
>>
>> That's not correct. File externals are supported (since 1.6 I think).
>> They have a couple of specific problems though (a lot of which have
>> been fixed in 1.7), because their implementation is entirely different
>> from directory externals (file externals use the "switch" mechanism
>> underneath, while dir externals are essentially an embedded checkout
>> with some sugar on top).
>>
>> So, concerning to the problem of the OP:
>>
>>> From: "Hutchinson, Steve (UK)" <steven.hutchinson_at_mbda-systems.com>
>>> To: "users_at_subversion.apache.org" <users_at_subversion.apache.org>
>>> Sent: Wednesday, May 1, 2013 7:01 AM
>>> Subject: SVN Externals 1.6 to 1.7 migration issue
>>>
>>> Hi,
>>>
>>> We're been using 1.6 svn externals to manage a FW task.
>>>
>>> In the repository we have a folder structure (which is pretty much defined
>>> by the tools we are using) as below :-
>>>
>>> FOLDER A -> file_a1.txt, file_a2.txt, file_a3.txt
>>> FOLDER B -> file_b1.txt, file_b2.txt, file_b3.txt
>>>
>>> Then using externals we link to those folder to create a WC PROJECT
>>> structure that looks like :-
>>>
>>> PROJECT -> FOLDER C -> file_a1.txt, file_a2.txt, file_a3.txt AND
>>> file_b1.txt
>>> PROJECT -> FOLDER D -> file_b1.txt, file_b2.txt, file_b3.txt
>>>
>>> The externals on PROJECT FOLDER we used looked like (not actually using file
>>> protocol, just created for example purposes) :-
>>>
>>> file:///D:/FPGA/SVN_ISSUE/repo/FOLDER_A FOLDER_C
>>> file:///D:/FPGA/SVN_ISSUE/repo/FOLDER_B FOLDER_D
>>> file:///D:/FPGA/SVN_ISSUE/repo/FOLDER_B/file_b1.txt FOLDER_C/file_b1.txt
>>>
>>> Worked ok with 1.6. We get the below error in 1.7 when updating of :-
>>>
>>> External failed: D:\FPGA\SVN_ISSUE\PROJECT\FOLDER_C\file_b1.txt
>>> Error: Cannot insert a file external defined on
>>> 'D:\FPGA\SVN_ISSUE\PROJECT' into the
>>> Error: working copy 'D:\FPGA\SVN_ISSUE\PROJECT\FOLDER_C'.
>>>
>>> We have tried a few things (will not share all at this point due to info
>>> overload), but wondered if there were any others that achieve something like
>>> this with 1.7 or could offer some advise ?
>>
>> So essentially you are "injecting" a file external (file_b1.txt)
>> inside a directory external (FOLDER_C (an external of FOLDER_A)).
>>
>> Hm, I remember some past discussions about edge cases like these
>> (during the working copy rewrite of 1.7), but I don't remember exactly
>> if this is actually a supported use case, or if it just happened to
>> work by accident in 1.6.
>>
>> Perhaps someone else on this list can shed some light on this?
>
> I had the following conversation with Philip Martin, on IRC (#svn-dev):
>
> <@jcorvel> philipm, perhaps you know the answer to this externals
> question (works in 1.6, but no longer in 1.7):
> http://svn.haxx.se/users/archive-2013-05/0006.shtml
> <philipm> A file external defined in one working copy but located
> inside another working copy is not going to work in 1.7 or 1.8.
> <philipm> The directory external is a separate working copy with its
> own .svn/wc.db and that knows nothing about the external defined in a
> different .svn/wc.db
> <@jcorvel> Ok. Sooo, is this more in the "it worked by accident in
> 1.6, but was never supposed to be supported" category? Or could this
> be considered a regression?
> <@jcorvel> I guess it doesn't matter ... semantics ...
> <philipm> I don't suppose we ever supported it deliberately.
> <@jcorvel> perhaps the more interesting question then is: is this at
> all supportable within the externals design (of wc-ng)? Or would this
> require "externals 2.0"?
> <philipm> If we change directory externals to be more like switch then
> the directory externals become part of a single working copy.
> <@jcorvel> ack
> <philipm> It may be relatively simple to do that. At present if you
> try to "svn sw URL path" and path does not exist then the switch
> fails.
> <philipm> But it only fails because of a high level check on path, in
> the past it was possible to switch a non-existant path.
> <philipm> (In the past being during 1.7 development)
> <philipm> If you switch a path that does not exist that looks very
> much like an external (it is how file externals are implemented)
> <philipm> One of the difficulties about externals is that the
> specification is so relaxed that there are so many different cases.
> <philipm> Almost any change is going to break some untested corner
> cases, while perhaps fixing other corner cases.
> <@jcorvel> philipm: so, realistically, this will probably not be fixed
> very soon (as in: in a 1.8.x release). At the very least, it needs (a
> lot of) work for coming up with a detailed spec etc ... to try and get
> a grip on the edge cases etc ...
> <philipm> I don't think it will be fixed in the near future. I don't
> know what we are going to do about externals.
>
>
> So I guess that answers it. It's not going to work in the near future.
> Short term I guess you have to find a workaround by structuring things
> differently. Longer term, someone needs to "champion" an effort to
> rework externals (starting by driving a discussion about it's
> specification etc ...). Volunteers welcome :-).
>

And some additional context was given by Bert Huijben:

<@Bert> jcorvel: Injecting file externals into a separate working copy
was deemed to be a never intended feature and a potential security
problem. It was explicitly disabled during 1.7 development.
<@Bert> jcorvel: The proper fix is part of the externals redesign to
something like viewspecs... where the file external just becomes a
working copy containing one file, but properly defined by some
ancestor
<@Bert> A problem with this 'feature' was that it was possible to
inject such externals, but not to remove them. The working copy that
contained the external didn't even contain the definition.

--
Johan