Thanks for this feedback, Bert. Several important points are emerging.
The first thing to get straight is what future we want for file
externals. File externals should behave like directory externals. The
directory externals system is far from a perfect solution to the
high-level challenges that users try to solve, but it has a lot of
precedent behind it now and is at least fairly logical. Despite us
having released file externals in 1.6 with several differences from
directory externals, I think absolutely all those differences should be
regarded as bugs and we should be bringing them back into line.
(If we do know of any way in which file externals behave "better" than
directory externals, that should be considered as a potential future
improvement for externals in general.)
In order to know what we're dealing with, we need to check and write
down the current behavioural differences, but in re-implementing file
externals handling, it's unquestionably the "directory externals"
behaviour that we need to be heading for.
On Fri, 2010-08-20, Bert Huijben wrote:
> > -----Original Message-----
> > From: Julian Foad [mailto:julian.foad_at_wandisco.com]
> > * An 'external' or 'switched' WC node is an immediate child of a
> > versioned WC directory. [3]
Changed to "parent or grandparent etc.".
> > * An 'external' or 'switched' WC node is attached to a repository
> > location other than the one implied by its WC parent directory.
>
> An external directory is not connected to it's parent. (You don't get
> it in the list of children using entries and/or nodes). To the parent
> it is just an unversioned directory, but if you look inside you find a
> working copy root that might (accidentally) be from the same
> repository. And it can be a few levels deeper than just an immediate
> child.
OK: an external directory is a WC root. I hadn't contemplated the
importance of that until now, but I think we can almost *define* the
required behaviour of externals in terms of that statement.
"An external directory is a nested WC that libsvn_client knows
something about: namely it knows how and when to create it and
to use it."
I'm suggesting "libsvn_client" here as an arbitrary but important level
of abstraction where a "complex WC" is implemented, and below which
there is a concept of "single WC" (single wc_id) that has no notion of
"externals".
We might choose to put this "complex WC" concept into libsvn_wc instead,
but it's useful for the conceptual distinction to exist somewhere in the
APIs, and this is probably a very good candidate for a chunk of
complexity that we would do well to keep out of libsvn_wc.
Let's see where that approach will take us.
[...]
> > * Client operations treat the node as part of its parent WC in some
> > ways, and treat it as special in other ways, depending on the
> > command and on user-specified options.
>
> Yes,
>
> Well, there are no real operations on external directories as they are
> just unversioned to the parent working copy.
At least two client operations, "svn update" and "svn status", recognize
externals and descend into them.
> External files... specialized code... specialized code... specialized
> code.
>
> External files are part of the parent working copy, so things like
> (recursive) locks and things apply on them. But these locks don't
> apply on directory externals as they are in a different working copy.
[...]
> > How should we represent external files and external dirs in the WC in a
> > unified manner? Do we need to explicitly mark them as 'external'? What
> > differences in behaviour should 'external' nodes and 'switched' nodes
> > exhibit within the WC?
(Heh, those were just meant to be examples of questions that this whole
topic might help to answer.)
> One option would be to pull file externals out of their parent working
> copy and make them their own working copy root. We can handle this in
> WC-DB. (Create a different wcroot id, defining the file to be its own
> root. Fix the update handling and we are done).
Yes!
If we are committed to making file externals like directory externals,
we need to re-specify a file external as "a separate WC consisting of a
single file". And when we try to do that, we have to re-think our
notion of a "WC root". A WC root will no longer always be a directory
and will not always be able to hold its own .svn admin area inside it.
Recall that the ability to check out and work on a single file from a
repository has been requested quite a few times in the life of
Subversion. That concept of a single-file checkout is alien to WC-1 and
to some assumptions built on our old implementation of WCs, but is not
in itself at all alien to the version control concepts of Subversion.
So a single-file WC is not specifically an "externals" concept, but
rather a general concept that will be useful in implementing externals.
> > Differences in WC State
> > =======================
> >
> > This table documents the differences in WC state at a high level.
> >
> > +----------------------+----------------------+----------------------+
> > | | switched | external |
> > +----------------------+----------------------+----------------------+
> > | | | |
> > | WC path affected | Existing child | Non-existent child |
>
> File external: existing child.
> > | | | |
> > | Target URL | Same repo | Same or other repo |
> File external: same repo
> > | | | |
> > | Pinned to a revision | No [2] | Optional; recorded |
> > | | | in 'svn:externals' |
> File external: recorded on the node itself.
Are you saying the pin revision is recorded on the node itself? As well
as in the 'svn:externals' property value of course?
> (And it's currently impossible to find the svn:externals definition
> for a specific external without looking at all it's ancestor
> directories)
Heh, that's a nuisance but not a logical difficulty.
> > | | | |
> > | Initial depth | Set by "switch" cmd | Infinity |
> > | | | |
> > +----------------------+----------------------+----------------------+
> >
> > ### Anything else?
> >
> >
> > Differences in Client Commands (Current behaviour)
> > ==================================================
> >
> > The following differences occur when a command encounters an 'external'
> > or 'switched' node during its operation. This table is to document the
> > current (1.6.x) behaviour.
> >
> > ### We'll need some test scripts to help gather this data.
> >
> > +----------------------+----------------------+----------------------+
> > | | switched | external |
> > +----------------------+----------------------+----------------------+
> > | | | |
> > | recursive descent, | yes | optional |
> > | in general | | |
> > | | | |
> > | status | report & descend | report [& descend] |
> > | | | |
> > | propget/list/etc. | ? | ? |
> yes | no
> > | | | |
> > | diff | ? | ? |
> yes | no
> > | | | |
> > | commit | descend | [descend?] |
> Yes | no (except for some buggy behavior relying on passing a limiting depth)
> > | | | |
> > | update | ? | ? |
> Yes | optional
> > | | | |
> > | switch | Can switch a WC root | No UI exists for an |
> > | | but it's not then | external outside |
> > | | recorded as such. | of a WC. |
> (externals definitions are normally updated after a switch by handling
> the svn:externals property changes)
> > | | | |
> > | ... | | |
>
> File externals are in all these cases handled as switch.
> > | | | |
> > +----------------------+----------------------+----------------------+
[...]
Updated in r988207. Thanks.
> > Footnotes
> > =========
> >
> > [2] The ability for "svn update -rX" to "pin" that part of the WC at
> > rX has been requested by users and seems like a reasonable (clean,
> > moderately useful) feature.
>
> I wouldn't want just using -r to pin the revision, but I certainly
> like the idea of a --pin option. And if we generalize this we can also
> use it to deny commits on pinned nodes. (Which is currently a feature
> request for file externals with a fixed revision)
Right. We *could* implement features such as denying commits on pinned
(file) externals right now, without generalizing the pinning in that
way, but it would be cleaner that way.
However, if we agree (as I think we do) that all externals should be
separate WCs, then that particular problem becomes less of an issue
because commit won't descend into them, at least not automatically.
> > [4] A possible exception is that "update" should obey the "pinned
> > revision number" if present. But that need not necessarily be an
> > exception: we might choose to define that functionality on all WC
> > nodes but only use it on 'external' nodes.
>
> See my answer on [2]. I like this generalized pinned idea as option to
> replace our checks on file externals.
>
> (First thing that comes to mind is how it would interact with 'svn
> switch on some parent')
That makes me think: the concept of pinning a WC subtree isn't just
"this needs to stay at revision R"
but rather
"this needs to stay pointing to $REPOS/RELPATH at revision R".
And, moving mentally sideways from the idea of "pinning to a revision"
towards "pinning to a tag", there is the important variant the "floating
tag":
"this needs to stay pointing to $REPOS/RELPATH at HEAD"
which is pretty much exactly what a revisionless svn:external property
says. But we don't have a strong requirement for this general pinning,
and the client can already tell from the svn:externals property what it
should do in the case of an external. I can't think of how it would be
*better* to implement it as a general WC feature at this time.
The focus of this thread is swinging towards how to unify file externals
with directory externals. And we're starting to find some powerful,
independent concepts which we can combine into a design, rather than
treating every requirement as a separate local problem. So far, we've
considered:
* The paired concepts of
- a "single WC" having a single wc_id, and
- "externals" as a way for a higher layer to link together
a set of "single WCs".
* The single-file checkout.
- useful in itself
- could be used to enable file externals
* The concept of "pinning" to a specific revision or node-rev.
- could be useful in itself
- could be used as a building block for pinned externals
- no need to implement this generalization at the moment;
it's just a thought to keep in mind.
- Julian
Received on 2010-08-23 19:20:48 CEST